Interactive virtual display

ABSTRACT

An interactive mid-air display including a display that presents a graphical user interface (GUI), optics projecting and rotating the GUI above the display such that the GUI is visible in-air in a plane rotated away from the display, a sensor including light emitters projecting beams towards the projected GUI, light detectors detecting reflections of the beams by objects interacting with the projected GUI, and a lens structure maximizing detection at each detector for light entering the lens structure at a respective location at a specific angle θ, whereby for each emitter-detector pair, maximum detection of light corresponds to the object being at a specific location in the projected GUI, in accordance with the locations of the emitter and detector and the angle θ, and a processor mapping detections of light for emitter-detector pairs to corresponding locations in the projected GUI, and translating the mapped locations to coordinates on the display.

CROSS REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/588,646, now U.S. Pat. No. 10,282,034, entitled TOUCH SENSITIVECURVED AND FLEXIBLE DISPLAYS, filed on May 7, 2017 by inventors ThomasEriksson, Alexander Jubner, Rozita Teymourzadeh, Stefan Holmgren, LarsSparf, Bengt Henry Hjalmar Edlund and Richard Berglind. U.S. patentapplication Ser. No. 15/588,646 claims priority benefit of U.S.Provisional Patent Application No. 62/425,087, entitled OPTICALPROXIMITY SENSOR AND ASSOCIATED USER INTERFACE, filed on Nov. 22, 2016by inventors Thomas Eriksson, Alexander Jubner, Rozita Teymourzadeh,Stefan Holmgren, Lars Sparf and Bengt Henry Hjalmar Edlund.

U.S. patent application Ser. No. 15/588,646 claims priority benefit ofU.S. Provisional Patent Application No. 62/462,034, entitled 3D IMAGINGWITH OPTICAL PROXIMITY SENSOR, filed on Feb. 22, 2017 by inventorsThomas Eriksson, Alexander Jubner, Rozita Teymourzadeh, Stefan Holmgren,Lars Sparf, Bengt Henry Hjalmar Edlund.

U.S. patent application Ser. No. 15/588,646 is a continuation-in-part ofU.S. patent application Ser. No. 14/960,369, now U.S. Pat. No.9,645,679, entitled INTEGRATED LIGHT GUIDE AND TOUCH SCREEN FRAME ANDMULTI-TOUCH DETERMINATION METHOD, filed on Dec. 5, 2015 by inventorsThomas Eriksson, Alexander Jubner, John Karlsson, Lars Sparf, SaskaLindfors and Robert Pettersson.

U.S. patent application Ser. No. 14/960,369 is a continuation of U.S.patent application Ser. No. 14/588,462, now U.S. Pat. No. 9,207,800,entitled INTEGRATED LIGHT GUIDE AND TOUCH SCREEN FRAME AND MULTI-TOUCHDETERMINATION METHOD, filed on Jan. 2, 2015 by inventors ThomasEriksson, Alexander Jubner, John Karlsson, Lars Sparf, Saska Lindforsand Robert Pettersson.

U.S. patent application Ser. No. 14/588,462 claims priority benefit ofU.S. Provisional Patent Application No. 62/054,353, entitled INTEGRATEDLIGHT GUIDE AND TOUCH SCREEN FRAME AND MULTI-TOUCH DETERMINATION METHOD,filed on Sep. 23, 2014 by inventors Saska Lindfors, Robert Pettersson,John Karlsson and Thomas Eriksson.

U.S. patent application Ser. No. 15/588,646, now U.S. Pat. No.9,921,661, is a continuation-in-part of U.S. patent application Ser. No.15/000,815, entitled OPTICAL PROXIMITY SENSOR AND ASSOCIATED USERINTERFACE, filed on Jan. 19, 2016 by inventors Thomas Eriksson,Alexander Jubner, Rozita Teymourzadeh, Håkan Sven Erik Andersson, PerRosengren, Xiatao Wang, Stefan Holmgren, Gunnar Martin Fröjdh, SimonFellin, Jan Tomas Hartman, Oscar Sverud, Sangtaek Kim, Rasmus Dahl-Örn,Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, ErikRosengren, John Karlsson, Remo Behdasht, Robin Kjell Åman, Joseph Shain,Oskar Hagberg and Joel Rozada.

U.S. patent application Ser. No. 15/000,815 claims priority benefitfrom:

-   -   U.S. Provisional Patent Application No. 62/107,536 entitled        OPTICAL PROXIMITY SENSORS and filed on Jan. 26, 2015 by        inventors Stefan Holmgren, Oscar Sverud, Sairam Iyer, Richard        Berglind, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren,        Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander        Jubner, Remo Behdasht, Simon Fellin, Robin Kjell Åman and Joseph        Shain;    -   U.S. Provisional Patent Application No. 62/197,813 entitled        OPTICAL PROXIMITY SENSOR and filed on Jul. 28, 2015 by inventors        Rozita Teymourzadeh, Håkan Sven Erik Andersson, Per Rosengren,        Xiatao Wang, Stefan Holmgren, Gunnar Martin Fröjdh and Simon        Fellin; and    -   U.S. Provisional Patent Application No. 62/266,011 entitled        OPTICAL PROXIMITY SENSOR and filed on Dec. 11, 2015 by inventors        Thomas Eriksson, Alexander Jubner, Rozita Teymourzadeh, Håkan        Sven Erik Andersson, Per Rosengren, Xiatao Wang, Stefan        Holmgren, Gunnar Martin Fröjdh, Simon Fellin and Jan Tomas        Hartman.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/630,737, now abandoned, entitledLIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed onFeb. 25, 2015 by inventors Thomas Eriksson and Stefan Holmgren.

U.S. patent application Ser. No. 14/630,737 is a continuation of U.S.patent application Ser. No. 14/140,635, now U.S. Pat. No. 9,001,087,entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE andfiled on Dec. 26, 2013 by inventors Thomas Eriksson and Stefan Holmgren.

U.S. patent application Ser. No. 14/140,635 is a continuation of U.S.patent application Ser. No. 13/732,456, now U.S. Pat. No. 8,643,628,entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE andfiled on Jan. 2, 2013 by inventors Thomas Eriksson and Stefan Holmgren.

U.S. patent application Ser. No. 13/732,456 claims priority benefit ofU.S. Provisional Patent Application No. 61/713,546, entitled LIGHT-BASEDPROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed on Oct. 14, 2012by inventor Stefan Holmgren.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/726,533, now U.S. Pat. No.9,678,601, entitled OPTICAL TOUCH SCREENS and filed on May 31, 2015 byinventors Robert Pettersson, Per Rosengren, Erik Rosengren, StefanHolmgren, Lars Sparf, Richard Berglind, Thomas Eriksson, Karl ErikPatrik Nordström, Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht.

U.S. patent application Ser. No. 14/726,533 is a continuation of U.S.patent application Ser. No. 14/311,366, now U.S. Pat. No. 9,063,614,entitled OPTICAL TOUCH SCREENS and filed on Jun. 23, 2014 by inventorsRobert Pettersson, Per Rosengren, Erik Rosengren, Stefan Holmgren, LarsSparf, Richard Berglind, Thomas Eriksson, Karl Erik Patrik Nordström,Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht.

U.S. patent application Ser. No. 14/311,366 is a continuation of PCTPatent Application No. PCT/US14/40579, entitled OPTICAL TOUCH SCREENSand filed on Jun. 3, 2014 by inventors Robert Pettersson, Per Rosengren,Erik Rosengren, Stefan Holmgren, Lars Sparf, Richard Berglind, ThomasEriksson, Karl Erik Patrik Nordström, Gunnar Martin Fröjdh, Xiatao Wangand Remo Behdasht.

U.S. patent application Ser. No. 15/588,646 is a continuation-in-part ofU.S. patent application Ser. No. 14/880,231, now U.S. Pat. No.10,004,985, entitled GAMING DEVICE and filed on Oct. 11, 2015 byinventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl ErikPatrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, JohnKarlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, SimonFellin, Robin Åman and Joseph Shain.

U.S. patent application Ser. No. 14/880,231 is a divisional of U.S.patent application Ser. No. 14/312,787, now U.S. Pat. No. 9,164,625,entitled OPTICAL PROXIMITY SENSORS and filed on Jun. 24, 2014 byinventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl ErikPatrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, JohnKarlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, SimonFellin, Robin Åman and Joseph Shain.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/555,731, now U.S. Pat. No.9,741,184, entitled DOOR HANDLE WITH OPTICAL PROXIMITY SENSORS and filedon Nov. 28, 2014 by inventors Sairam Iyer, Stefan Holmgren and PerRosengren.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/791,414, now U.S. Pat. No.10,324,565, entitled OPTICAL PROXIMITY SENSOR FOR TOUCH SCREEN ANDASSOCIATED CALIBRATION TOOL and filed on Jul. 4, 2015 by inventors PerRosengren, Xiatao Wang and Stefan Holmgren.

U.S. patent application Ser. No. 14/312,787 is a continuation-in-part ofU.S. patent application Ser. No. 13/775,269, now U.S. Pat. No.8,917,239, entitled REMOVABLE PROTECTIVE COVER WITH EMBEDDED PROXIMITYSENSORS and filed on Feb. 25, 2013 by inventors Thomas Eriksson, StefanHolmgren, John Karlsson, Remo Behdasht, Erik Rosengren and Lars Sparf.

U.S. patent application Ser. No. 14/312,787 is also a continuation ofPCT Patent Application No. PCT/US14/40112, entitled OPTICAL PROXIMITYSENSORS and filed on May 30, 2014 by inventors Stefan Holmgren, SairamIyer, Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, PerRosengren, Erik Rosengren, John Karlsson, Thomas Eriksson, AlexanderJubner, Remo Behdasht, Simon Fellin, Robin Åman and Joseph Shain.

PCT Patent Application No. PCT/US14/40112 claims priority benefit from:

-   -   U.S. Provisional Patent Application No. 61/846,089, entitled        PROXIMITY SENSOR FOR LAPTOP COMPUTER AND ASSOCIATED USER        INTERFACE and filed on Jul. 15, 2013 by inventors Richard        Berglind, Thomas Eriksson, Simon Fellin, Per Rosengren, Lars        Sparf, Erik Rosengren, Joseph Shain, Stefan Holmgren, John        Karlsson and Remo Behdasht;    -   U.S. Provisional Patent Application No. 61/838,296, entitled        OPTICAL GAME ACCESSORIES USING REFLECTED LIGHT and filed on Jun.        23, 2013 by inventors Per Rosengren, Lars Sparf, Erik Rosengren,        Thomas Eriksson, Joseph Shain, Stefan Holmgren, John Karlsson        and Remo Behdasht; and    -   U.S. Provisional Patent Application No. 61/828,713, entitled        OPTICAL TOUCH SCREEN SYSTEMS USING REFLECTED LIGHT and filed on        May 30, 2013 by inventors Per Rosengren, Lars Sparf, Erik        Rosengren and Thomas Eriksson.        The contents of these applications are hereby incorporated by        reference in their entireties.

FIELD OF THE INVENTION

The field of the present invention is curved and flexible touch surfacesystems using optical touch sensors.

BACKGROUND OF THE INVENTION

Large curved displays reduce outer edge distortions and provide apanoramic view. Curved display screens are also used in certain mobilephones. One of the issues in the use of curved screens in commercialelectronics is how accurately it can work with a touch sensor.

A flexible display is an electronic visual display which is flexible innature; as opposed to the more prevalent traditional flat screendisplays used in most electronics devices. In recent years there hasbeen a growing interest from numerous consumer electronics manufacturersto apply this display technology in e-readers, mobile phones and otherconsumer electronics (—Wikipedia, “Flexible display”). The flexiblenature of the display prevents manufacturers from adding conventionaltouch sensors to flexible displays.

The Wi-Fi Alliance® launched the Miracast® certification program at theend of 2012. WI-FI ALLIANCE and MIRACAST are registered trademarks ofWI-FI Alliance Corporation California. Devices that areMiracast-certified can communicate with each other, regardless ofmanufacturer. Adapters are available that plug into High-DefinitionMultimedia Interface (HDMI) or Universal Serial Bus (USB) ports,allowing devices without built-in Miracast support to connect viaMiracast. Miracast employs the peer-to-peer Wi-Fi Direct standard tosend video and audio. WI-FI DIRECT is a registered trademark of WI-FIAlliance Corporation California. IPv4 is used on the Internet layer. Onthe transport layer, TCP or UDP are used. On the application layer, thestream is initiated and controlled via RTSP, RTP for the data transfer.

The present invention enables touch and gesture input on aMiracast-connected TV, monitor or projector to be detected andcommunicated back to the server, laptop, tablet or smartphone thatoriginally sent the displayed image.

SUMMARY

There is thus provided in accordance with an embodiment of the presentinvention a touch system having a curved touch surface, including ahousing, a curved surface near the housing, light emitters mounted inthe housing projecting light beams out of the housing over and acrossthe curved surface, such that at least some of the light beams areincident upon and reflected by the curved surface when crossing over thecurved surface, light detectors mounted in the housing detectingreflections, by a reflective object touching the curved surface, of thelight beams projected by the light emitters, lenses mounted and orientedin the housing relative to the light emitters and to the light detectorssuch that (i) there is a particular angle of entry at which each lightdetector receives a maximal light intensity when light beams enter alens corresponding to the light detector at the particular angle ofentry, and (ii) there are target positions, associated withemitter-detector pairs, on the curved surface, whereby for eachemitter-detector pair, when the object is located at the target positionassociated with the emitter-detector pair, then light beams emitted bythe light emitter of that pair are reflected by the object into the lenscorresponding to the light detector of that pair at the particular angleof entry, and a processor connected to the light emitters and to thelight detectors, synchronously co-activating emitter-detector pairs, andcalculating a location of the object touching the curved surface bydetermining an emitter-detector pair among the co-activatedemitter-detector pairs, for which the light detector of the pair detectsa maximal amount of light, and by identifying the target positionassociated with the pair.

In certain embodiments of the invention, the curved surface is aretractable surface.

In certain embodiments of the invention, the curved surface includes afirst portion that is flat and a second portion that is curved, thesecond portion being further from the housing than the first portion,and when the object touches the second portion of the curved surface,some of the light reflected by the object is incident upon and reflectedby the curved surface while crossing the curved surface toward the lightdetectors.

In certain embodiments of the invention, the processor is configured tocalculate the location of the object by additionally determiningpositions associated with co-activated emitter-detector pairs thatneighbor the thus-identified position, and calculating a weightedaverage of the thus-identified position and the thus-determinedneighboring positions, wherein each position's weight in the averagecorresponds to a degree of detection of the reflected light beam for theemitter-detector pair to which that position is associated.

There is additionally provided in accordance with an embodiment of thepresent invention a touch system having a flexible touch surface,including a housing, a flexible surface near the housing, light emittersmounted in the housing projecting light beams out of the housing overand across the flexible surface such that, when the flexible surface isconcavely flexed, at least some of the light beams are incident upon andreflected by the flexible surface as they cross over the flexiblesurface, light detectors mounted in the housing detecting reflections,by a reflective object touching the flexible surface, of the light beamsprojected by the emitters, lenses mounted and oriented in the housingrelative to the light emitters and to the light detectors such that (i)there is a particular angle of entry at which each light detectorreceives a maximal light intensity when light beams enter a lenscorresponding to the light detector at the particular angle of entry,and (ii) there are target positions, associated with emitter-detectorpairs, on the flexible surface, whereby for each emitter-detector pair,when the object is located at the target position associated with theemitter-detector pair, then light beams emitted by the light emitter ofthat pair are reflected by the object into the lens corresponding to thelight detector of that pair at the particular angle of entry, and aprocessor connected to the light emitters and to the light detectors,synchronously co-activating emitter-detector pairs, and calculating alocation of the object touching the flexible surface by determining anemitter-detector pair among the co-activated emitter-detector pairs, forwhich the light detector of the pair detects a maximal amount of light,and by identifying the target position associated with the pair.

In certain embodiments of the invention, the flexible surface isretractable into the housing.

In certain embodiments of the invention, when the flexible displayscreen is concavely curved, and the object is touching a curved portionof the flexible surface, some of the light reflected by the object isincident upon and reflected by the flexible surface while crossing theflexible surface toward the light detectors.

In certain embodiments of the invention, the processor is configured tocalculate a location of the object by additionally determining positionsassociated with co-activated emitter-detector pairs that neighbor thethus-identified position, and calculating a weighted average of thethus-identified position and the thus-determined neighboring positions,wherein each position's weight in the average corresponds to a degree ofdetection of the reflected light beam for the emitter-detector pair towhich that position is associated.

There is further provided in accordance with an embodiment of thepresent invention a method of generating a three-dimensional image of anobject using an optical proximity sensor in the shape of a bar, thesensor including a linear array of interleaved individually activatablelight emitters and photodiode detectors mounted in the bar, a pluralityof lenses through which light emitted by the emitters is projected intoa planar airspace outside the bar, and through which light reflected byan object in the planar airspace is projected onto the photodiodedetectors, wherein each lens is paired with: (i) a respective one of theemitters, to maximize outgoing light emission in a specific direction atan angle, designated θ, relative to the bar, the angle θ being the samefor each lens-emitter pair, and (ii) respective first and second ones ofthe photodiode detectors, to maximize incoming reflected light detectionat respective first and second specific directions at respective angles,designated φ₁ and φ₂, relative to the bar, the angle φ₁ being the samefor each first detector-lens pair, and the angle φ₂ being the same foreach second detector-lens pair, the method including repeatedly movingthe proximity sensor, such that light emitted by the emitters isprojected into a different planar airspace after each move, repeatedlyselectively activating the emitters and the photodiode detectors,repeatedly identifying locations on the object in the planar airspace,based on outputs of the photodiode detectors and the known angles θ, φ₁and φ₂, and combining the identified locations to generate athree-dimensional image of the object, based on the orientations of thedifferent planar airspaces.

In certain embodiments of the invention, the method further includesrepeatedly identifying a shape of the object in the planar airspace,based on the identified locations, and combining the identified shapesto generate a three-dimensional image of the object, based on theorientations of the different planar airspaces.

In certain embodiments of the invention, the combining of the identifiedlocations creates a point cloud of the object.

In certain embodiments of the invention, the method further includesrepeatedly identifying a size of the object in the planar airspace,based on outputs of the photodiode detectors and the known angles θ, φ₁and φ₂, and combining the identified sizes in the different planarairspaces to derive a three-dimensional volume that contains the object,based on orientations of the planar airspaces.

In certain embodiments of the invention, the method further includesrepeatedly identifying a distance between the object and the bar in theplanar airspace, based on outputs of the photodiode detectors and theknown angles θ, φ₁ and φ₁, and combining the identified distances in thedifferent planar airspaces to derive a location of the object, based onorientations of the planar airspaces.

In certain embodiments of the invention, the light emitters are laserdiodes.

In certain embodiments of the invention, the sensor bar further includesat least one inertial measurement unit (IMU), the method furtherincluding repeatedly identifying the orientation of the planar airspacebased on outputs of the at least one IMU.

There is yet further provided in accordance with an embodiment of thepresent invention a handheld 3D scanner including a housing that can belifted and moved by a human hand, a linear array of interleavedindividually activatable light emitters and photodiode detectors mountedin the housing, a plurality of lenses, mounted in the housing, throughwhich light emitted by the emitters is projected into a planar airspaceoutside the housing, and through which light reflected by the object inthe planar airspace is directed onto the photodiode detectors, whereineach lens is paired with: (i) a respective one of the emitters, tomaximize outgoing light emission in a specific direction at an angle,designated θ, relative to the lens, the angle θ being the same for eachlens-emitter pair, and (ii) respective first and second ones of thephotodiode detectors, to maximize incoming reflected light detection atrespective first and second specific directions at respective angles,designated φ₁ and φ₂, relative to the lens, the angle φ₁ being the samefor each first detector-lens pair, and the angle φ₂ being the same foreach second detector-lens pair, at least one inertial measurement unitmounted in the housing and tracking the changing orientations of theplanar airspace when the housing is lifted and moved, a processor (i)connected to the emitters and photodiode detectors repeatedlyselectively activating the emitters and the photodiode detectors,repeatedly identifying a shape of the object in the planar airspace,based on outputs of the photodiode detectors and the known angles θ, φ₁and φ₂, (ii) connected to the inertial measurement unit, and (iii)combining the identified shapes to generate a three-dimensional image ofthe object, based on the tracked orientations of the planar airspaces.

There is moreover provided in accordance with an embodiment of thepresent invention a handheld 3D scanner including a housing that can belifted and moved by a human hand, a linear array of interleavedindividually activatable light emitters and photodiode detectors mountedin the housing, a plurality of lenses, mounted in the housing, throughwhich light emitted by the emitters is projected into a planar airspaceoutside the housing, and through which light reflected by the object inthe planar airspace is directed onto the photodiode detectors, whereineach lens is paired with: (i) a respective one of the emitters, tomaximize outgoing light emission in a specific direction at an angle,designated θ, relative to the lens, the angle θ being the same for eachlens-emitter pair, and (ii) respective first and second ones of thephotodiode detectors, to maximize incoming reflected light detection atrespective first and second specific directions at respective angles,designated φ₁ and φ₂, relative to the lens, the angle φ₁ being the samefor each first detector-lens pair, and the angle φ₂ being the same foreach second detector-lens pair, at least one inertial measurement unitmounted in the housing tracking the changing orientations of the planarairspace when the housing is lifted and moved, a processor (i) connectedto the emitters and photodiode detectors repeatedly selectivelyactivating the emitters and the photodiode detectors, repeatedlyidentifying locations on the object in the planar airspace, based onoutputs of the photodiode detectors and the known angles θ, φ₁ and φ₂,(ii) connected to the inertial measurement unit, and (iii) combining theidentified locations to generate a three-dimensional point cloud of theobject, based on the tracked orientations of the planar airspaces.

There is additionally provided in accordance with an embodiment of thepresent invention a scanning system including a rotating stand on whichan object is placed, a linear array of interleaved individuallyactivatable light emitters and photodiode detectors, a plurality oflenses through which light emitted by the emitters is projected into aplanar airspace above the rotating stand and through which lightreflected by the object in the planar airspace is directed onto thephotodiode detectors, wherein each lens is paired with: (i) a respectiveone of the emitters, to maximize outgoing light emission in a specificdirection at an angle, designated θ, relative to the lens, the angle θbeing the same for each lens-emitter pair, and (ii) respective first andsecond ones of the photodiode detectors, to maximize incoming reflectedlight detection at respective first and second specific directions atrespective angles, designated φ₁ and φ₂, relative to the lens, the angleφ₁ being the same for each first detector-lens pair, and the angle φ₂being the same for each second detector-lens pair, a motor connected tothe stand incrementally rotating the object, such that light emitted bythe emitters is reflected by a different portion of the object aftereach move, a processor connected to the emitters and the photodiodedetectors repeatedly selectively activating the emitters and thephotodiode detectors, repeatedly identifying a shape of the object inthe planar airspace, based on outputs of the photodiode detectors andthe known angles θ, φ₁ and φ₂, and combining the identified shapes togenerate a three-dimensional image of the object, based on thoseportions of the object that intersect the planar airspaces.

There is further provided in accordance with an embodiment of thepresent invention a fitting room system including the scanning systemdescribed in the previous paragraph, wherein the object is a shopper forclothes, wherein the processor extracts body measurements of the shopperfrom the three-dimensional image of the object, wherein the processor iscommunicatively coupled to a database of clothes indexed by at least oneof the body measurements, and wherein the processor runs a query on thedatabase that returns database items that match at least one of the bodymeasurements, and a display connected to the processor displaying thedatabase items returned by the query.

In certain embodiments of the fitting room system, the processor createsan avatar of the shopper based on the body measurements and renders animage of the avatar wearing the database items returned by the query.

There is yet further provided in accordance with an embodiment of thepresent invention an avatar generator including the scanning systemdescribed hereinabove wherein the processor creates an avatar of theobject based on the three-dimensional image of the object.

In certain embodiments of the avatar generator, the processor isconnected to a network, and the processor outputs the avatar over thenetwork to a computer storing a user profile to which the avatar isadded.

There is moreover provided in accordance with an embodiment of thepresent invention a touch pad for determining locations of multipleobjects concurrently touching the pad, including a housing, an exposedsurface mounted in the housing, two proximity sensor bars mounted in thehousing along different edges of the exposed surface, each proximitysensor bar including a plurality of light pulse emitters projectinglight out of the housing along a detection plane over and parallel tothe exposed surface, a plurality of light detectors detectingreflections of the light projected by the emitters, by a reflectiveobject passing through the detection plane, a plurality of lensesoriented relative to the emitters and the detectors in such a mannerthat each emitter-detector pair has a target position in the detectionplane associated therewith, the target position being such that when theobject is located at the target position, light emitted by the emitterof that pair passes through one of the lenses and is reflected by theobject back through one of the lenses to the detector of that pair,wherein the target positions associated with emitter-detector pairs ofeach of the proximity sensor bars comprise some common positions, and aprocessor mounted in the housing and connected to the emitters and tothe detectors of the two proximity sensor bars, the processorsynchronously co-activating respective emitter-detector pairs of the twoproximity sensor bars that are associated with common target positions,and determining locations of multiple objects that are concurrentlypassing through the detection plane.

In certain embodiments of the touch pad invention, the proximity sensorbars are mounted along adjacent edges of the exposed surface.

In certain embodiments of the touch pad invention, the proximity sensorbars are mounted along opposite edges of the exposed surface.

There is additionally provided in accordance with an embodiment of thepresent invention an interactive computer system including a firstprocessor configured (i) to render a graphical user interface (GUI), and(ii) to respond to input to the GUI, a display device that renders theGUI on a surface, a first wireless transmitter and receiver transmittingthe GUI from the first processor to the display device, wherein thetransmitter is connected to the first processor and the receiver isconnected to the display device, a proximity sensor, including a housingmounted along an edge of the surface, a plurality of light emittersmounted in the housing operable when activated to project light out ofthe housing over the surface, a plurality of light detectors mounted inthe housing operable when activated to detect amounts of arriving light,and a second processor connected to the emitters and to the detectors,configured (i) to selectively activate the light emitters and the lightdetectors, and (ii) to identify the location of an object touching thesurface, based on amounts of light detected by the activated detectorswhen the object reflects light projected by the activated light emittersback into the housing, a second wireless transmitter and receivertransmitting the identified location to the first processor as input tothe GUI, wherein the transmitter is connected to the second processorand the receiver is connected to the first processor.

In certain embodiments of the computer system, the first processor, thefirst transmitter and the second receiver are mounted in a mobile phone,and the display device is one of the group consisting of: a TV, acomputer monitor and a projector.

In certain embodiments of the computer system, the first processor andthe first transmitter are mounted in a mobile phone, the second receiveris mounted in a dongle that is removably connected to the mobile phoneand wherein the display device is one of the group consisting of: a TV,a computer monitor and a projector.

In certain embodiments of the computer system, the first processor, thefirst transmitter and the second receiver comprise a server, and thedisplay device is a thin client.

There is further provided in accordance with an embodiment of thepresent invention an interactive computer system including a display, ahousing mounted along an edge of the display, a proximity sensor mountedin the housing, including a plurality of light emitters operable whenactivated to project light out of the housing over the display, and aplurality of light detectors operable when activated to detect amountsof arriving light, a processor mounted in the housing configured (i) torender a graphical user interface (GUI), (ii) to selectively activatethe light emitters and the light detectors, and (iii) to identify one ormore locations of an object touching the display, based on amounts oflight detected by the activated detectors when the object reflects lightprojected by the activated light emitters back into the housing, and awireless transmitter and receiver transmitting the GUI from theprocessor to the display, wherein the transmitter is connected to theprocessor and the receiver is connected to the display and wherein theprocessor is further configured to respond to the identified one or morelocations as input to the GUI.

There is yet further provided in accordance with an embodiment of thepresent invention an optical assembly for detecting locations of objectsin any of multiple parallel spatial planes, featuring areflectance-based sensor that emits light into a single detection planeof the sensor and detects reflections of the emitted light, reflected byan object located in the detection plane of the sensor, a lightre-director positioned away from the sensor that re-directs lightemitted by the sensor into one or more spatial planes parallel to thedetection plane of the sensor and, when the object is located in the oneor more spatial planes, re-directs light reflected by the object intothe detection plane of the sensor, and a processor connected to thesensor that controls light emitted by the sensor and, when an objectpasses through one or more of the spatial planes, the processoridentifies both (i) the spatial planes through which the object passes,and (ii) the location of the object within the spatial planes throughwhich it passes.

In certain embodiments of the optical assembly, the reflectance-basedsensor includes an array of interleaved light emitters and photodiodes.

In certain embodiments of the optical assembly, the reflectance-basedsensor includes a time-of-flight sensor.

In certain embodiments of the optical assembly, the reflectance-basedsensor includes two cameras.

In certain embodiments of the optical assembly, the light re-director isa folding mirror.

In certain embodiments of the optical assembly, the light re-director isa beam splitter.

There is moreover provided in accordance with an embodiment of thepresent invention a method for detecting locations of objects in any ofmultiple parallel spatial planes, featuring: providing an opticalassembly including (a) a reflectance-based sensor that emits light intoa single detection plane of the sensor and detects reflections of theemitted light, reflected by an object located in the detection plane ofthe sensor, and (b) a light re-director positioned away from the sensorthat re-directs light emitted by the sensor into one or more spatialplanes parallel to the detection plane of the sensor and, when theobject is located in the one or more spatial planes, re-directs lightreflected by the object into the detection plane of the sensor,providing a processor connected to the sensor that controls lightemitted by the sensor and processes light detected by the sensor, andwhen an object passes through one or more of the spatial planes,detecting, by the processor, both (i) the spatial planes through whichthe object passes, and (ii) the location of the object within thespatial planes through which it passes, including: detecting, by theprocessor, one or more virtual locations of the object within thedetection plane of the sensor, based on light reflected by the objectthat is re-directed to the detection plane of the sensor and detected bythe sensor, and transforming, by the processor, the one or more virtuallocations of the object within the detection plane of the sensor tocorresponding one or more real locations of the object within one ormore spatial planes parallel to the detection plane of the sensor, basedon the position of the light re-director relative to the sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be more fully understood and appreciated fromthe following detailed description, taken in conjunction with thedrawings in which:

FIG. 1 is a simplified illustration of light being emitted from lightsources along a solid line, and being reflected along dashed lines tolight sensors, in accordance with an embodiment of the presentinvention;

FIG. 2 is an illustration of backward and forward hotspot signal values,i.e., signal values for source/sensor pairs, in accordance with anembodiment of the present invention;

FIGS. 3-5 are illustrations showing a signal value relationship betweenneighboring hotspots within 3×3 grids of forward and backward hotspots,in accordance with an embodiment of the present invention

FIG. 6 is an illustration showing a relationship between two signalvalues v0 and v1 (solid lines) expressed as r=log(v1)−log(v0) (dashedline), in accordance with an embodiment of the present invention;

FIG. 7 is an illustration showing use of triangles to mark areassurrounding hotspots at which an object generates strong signal values,in accordance with an embodiment of the present invention;

FIG. 8 is an illustration of detection errors across a 100×64 mm touchscreen, in accordance with an embodiment of the present invention;

FIG. 9 is an illustration of a 2D histogram of sample error vectors, inaccordance with an embodiment of the present invention;

FIGS. 10-13 are simplified illustrations of a proximity sensor, inaccordance with an embodiment of the present invention;

FIGS. 14 and 15 are simplified illustrations of calibration tools forthe proximity sensor of FIGS. 10-13, in accordance with an embodiment ofthe present invention;

FIGS. 16 and 17 are simplified illustrations showing that thecalibration tool of FIG. 15 identifies how the emitters and detectors ofthe proximity sensor have been mounted therein, in accordance with anembodiment of the present invention;

FIG. 18 is a simplified illustration showing a proximity sensordetecting a proximal object, in accordance with an embodiment of thepresent invention;

FIG. 19 is a simplified illustration of a two-dimensional image ofdetection values, in accordance with an embodiment of the presentinvention;

FIGS. 20 and 21 are simplified illustrations showing a detectedreflection value for an emitter-receiver pair that is not associatedwith that pair's hotspot location, in accordance with an embodiment ofthe present invention;

FIG. 22 is a simplified illustration of a detected partial circumferenceof an object in a two-dimensional image of detection values, inaccordance with an embodiment of the present invention;

FIG. 23 is a simplified illustration of a method of estimating a partialcircumference of an object, in accordance with an embodiment of thepresent invention;

FIG. 24 is an illustration of a saddle roof or hyperbolic paraboloidcorresponding to the emitter light paths and reflected light paths of a3D proximity sensor, in accordance with an embodiment of the presentinvention;

FIG. 25 is a simplified illustration of a circular arrangement of sixemitters and six receivers arranged alternatingly along a circular basethat provide 30 hotpot locations along a 3D hyperboloid, in accordancewith an embodiment of the present invention;

FIG. 26 is a simplified illustration of a grid representing emitted andreflected light beams from a circular arrangement of 16 emitters and 16receivers arranged alternatingly along the circle, which provide 176hotpot locations along a 3D hyperboloid, in accordance with anembodiment of the present invention;

FIGS. 27-29 are simplified illustrations of a first input accessory forvirtual reality systems, in accordance with an embodiment of the presentinvention;

FIGS. 30 and 31 are simplified illustrations of a second input accessoryfor virtual reality systems, in accordance with an embodiment of thepresent invention;

FIGS. 32-34 are simplified illustrations of a third input accessory forvirtual reality systems, in accordance with an embodiment of the presentinvention;

FIGS. 35-37 are simplified illustrations of an interactive mirror, inaccordance with an embodiment of the present invention;

FIGS. 38 and 39 are simplified illustrations of an interactive mirrorwith an optical sensor mounted behind the mirror, in accordance with anembodiment of the present invention;

FIG. 40 is a simplified illustration of an interactive mid-air displaysystem, in accordance with an embodiment of the present invention;

FIG. 41 is a simplified illustration showing an upright proximity sensorbar being repeatedly moved in relation to a detected object, to generatea three-dimensional image of the object, in accordance with anembodiment of the present invention;

FIG. 42 is a simplified illustration showing a horizontal proximitysensor bar being repeatedly moved in relation to a detected object, togenerate a three-dimensional image of the object, in accordance with anembodiment of the present invention;

FIG. 43 is an illustration of a scanned cup and scan data combined intoa 3D model, in accordance with an embodiment of the present invention;

FIG. 44 is a simplified illustration showing three fingers being scannedby a proximity sensor on four different sides of the fingers, inaccordance with an embodiment of the present invention;

FIG. 45 is a simplified illustration showing detections of the scanoperations illustrated in FIG. 44, in accordance with an embodiment ofthe present invention;

FIG. 46 is a simplified illustration showing signal intensities in thescan operations illustrated in FIG. 44, in accordance with an embodimentof the present invention;

FIG. 47 is a simplified illustration of a debugger interface unitscanned by a proximity sensor on four different sides of the unit, inaccordance with an embodiment of the present invention;

FIG. 48 is a simplified illustration showing signal intensities in thescan operations illustrated in FIG. 47, in accordance with an embodimentof the present invention;

FIG. 49 is a simplified illustration showing a proximity sensor directedabove an object, and the resulting scan information, in accordance withan embodiment of the present invention;

FIG. 50 is a simplified illustration showing a proximity sensor directedat an upper edge of an object, and the resulting scan information, inaccordance with an embodiment of the present invention;

FIG. 51 is a simplified illustration showing a proximity sensor directedat the center of an object, and the resulting scan information, inaccordance with an embodiment of the present invention;

FIG. 52 is a graph of detection signals for individual emitter-detectorpairs generated by 11 scan sequences performed by a proximity sensorsituated at a fixed location in relation to an object, in accordancewith an embodiment of the present invention;

FIG. 53 is a simplified illustration showing a proximity sensor scanningan object at four different distances from the object, in accordancewith an embodiment of the present invention;

FIG. 54 is a radar plot of detection signals generated by scan sequencesperformed at multiple locations around an object, and at differentdistances from the object, in accordance with an embodiment of thepresent invention;

FIG. 55 is a 3D graph of signal intensities generated by a proximitysensor bar placed along four sides of a scanned object at a firstheight, in accordance with an embodiment of the present invention;

FIG. 56 is a graph of detection signals generated by scan sequencesperformed at multiple locations around an object, similar to those ofFIG. 55 but at a second fixed height in relation to the object, inaccordance with an embodiment of the present invention;

FIG. 57 is a 3D graph of signal intensities generated by a proximitysensor bar placed along the four sides of a scanned object at a thirdheight, in accordance with an embodiment of the present invention;

FIG. 58 is a 3D graph of signal intensities generated by a proximitysensor bar placed along the four sides of a scanned object at a fourthheight, in accordance with an embodiment of the present invention;

FIG. 59 is a simplified illustration of a 3D image of a scanned object,generated by combining detections of a proximity sensor placed atlocations around the object, and at different heights in relation to theobject, in accordance with an embodiment of the present invention;

FIGS. 60-63 are illustrations showing translating the location of anobject, as detected in a single proximity sensor's detection plane, to acommon detection plane formed by the union of multiple proximity sensordetection planes, in accordance with an embodiment of the presentinvention;

FIG. 64 is a simplified flow chart of a process for converting scaninformation into a 3D representation of a scanned object, in accordancewith an embodiment of the present invention;

FIGS. 65-70 are illustrations of scanned objects and 3D images of thoseobjects generated from proximity sensor detections, in accordance withan embodiment of the present invention;

FIG. 71 is a simplified flow chart of a process for converting scaninformation into a 3D representation of a scanned object, in accordancewith an embodiment of the present invention;

FIG. 72 is an illustration showing translating an object's bounding boxwithin a shared detection plane, in accordance with an embodiment of thepresent invention;

FIG. 73 is a simplified flow chart for a method of generating a 3D imageof a scanned object, by combining detections of a proximity sensorplaced at locations around the object, and at different heights inrelation to the object, in accordance with an embodiment of the presentinvention;

FIGS. 74 and 75 are illustrations of scanned objects and 3D images ofthose object generated from proximity sensor detections, in accordancewith an embodiment of the present invention;

FIG. 76 is an illustration of a scanned scotch tape dispenser and theheights of that dispenser detected on four sides thereof, in accordancewith an embodiment of the present invention;

FIGS. 77 and 78 are illustrations of signal behavior as a fingertipmoves between hotspots, in accordance with an embodiment of the presentinvention;

FIGS. 79-81 are illustrations showing measurement of errors by aproximity sensor, for a plurality of different objects, in accordancewith an embodiment of the present invention;

FIG. 82 is a simplified illustration of a system for scanning a humanbody for use in retail clothing applications and for generating avatarsused in social media apps and in games, in accordance with an embodimentof the present invention;

FIG. 83 is an illustration showing a situation in which a second objectin a proximity sensor detection plane is not detected;

FIG. 84 is an illustration showing how two proximity sensor bars alongdifferent edges of a detection plane are used to solve the problem ofFIG. 83, in accordance with an embodiment of the present invention;

FIG. 85 is a simplified illustration showing some of the projected andreflected light paths used by the two proximity sensor bars of FIG. 84along different edges of the detection plane, in accordance with anembodiment of the present invention;

FIG. 86 is a simplified illustration of a touch screen and an L-shapedproximity sensor bar for placement along adjacent edges of the screen,in accordance with an embodiment of the present invention;

FIG. 87 is an exploded view of a touch screen assembly featuring anL-shaped proximity sensor bar along adjacent edges of the screen, inaccordance with an embodiment of the present invention;

FIG. 88 is a cutaway view of the touch screen assembly of FIG. 87,showing paths of projected and reflected light beams into and out of oneof the proximity sensor bars, in accordance with an embodiment of thepresent invention;

FIG. 89 shows three simplified flowcharts of methods of activating twoproximity sensor bars along different edges of a touch surface, inaccordance with an embodiment of the present invention;

FIG. 90 is an illustration of a wireless input-output system, inaccordance with an embodiment of the present invention;

FIG. 91 is a simplified block diagram of a wireless input-output systemfeaturing a mobile phone, a TV and a proximity sensor bar, in accordancewith an embodiment of the present invention;

FIG. 92 is a simplified block diagram of a wireless input-output systemfeaturing a server, a thin client and a proximity sensor bar, inaccordance with an embodiment of the present invention;

FIG. 93 is a simplified block diagram of a wireless input-output systemfeaturing a thin client and a proximity sensor bar, in accordance withan embodiment of the present invention;

FIGS. 94 and 95 are simplified illustrations of a curved display andproximity sensing light-beams used to detect gestures thereon, inaccordance with an embodiment of the present invention;

FIGS. 96-98 are simplified illustrations of a flexible display andproximity sensing light-beams used to detect gestures thereon, inaccordance with an embodiment of the present invention;

FIGS. 99 and 100 are simplified illustrations of a housing holding aretractable display and a proximity sensor that detects gestures on theretractable display when the display is drawn out of the housing, inaccordance with an embodiment of the present invention;

FIG. 101 is a simplified illustration of a first proximity sensingsystem that detects objects in multiple parallel detection planes, inaccordance with an embodiment of the present invention;

FIGS. 102 and 103 are simplified illustrations of mapping detections indifferent detection planes in the proximity sensing system of FIG. 101,in accordance with an embodiment of the present invention;

FIG. 104 is a simplified illustration of a second proximity sensingsystem that detects objects in multiple parallel detection planes, inaccordance with an embodiment of the present invention;

FIGS. 105 and 106 are simplified illustrations of a keyboard using theproximity sensing system of FIG. 104 to detect objects above the keysand also detect depressed keys, in accordance with an embodiment of thepresent invention; and

FIGS. 107 and 108 are simplified illustrations of mapping detectionsabove the keys and detections of depressed keys in the keyboard of FIGS.105 and 106, in accordance with an embodiment of the present invention.

The following table catalogs the numbered elements and lists the figuresin which each numbered element appears. Similarly numbered elementsrepresent elements of the same type, but they need not be identicalelements.

Numbered Elements Element Description Figures 101-110 emitter 10-18, 20,21, 23, 25, 91-95, 103  201-211 photo-detector 10-18, 20, 21, 23, 25,91-95, 103 301-304 lens 10-15, 17, 18, 20, 21 305 reflectors 38 306 lens39 307 refractive surface 39 308 reflective surface 39 309 light guide87, 88 401, 404, 407, 421, 426-429 emitted light beam 1, 10-18, 20, 21,25, 83, 84, 88, 94-98, 103-106 402, 403, 405, 406, 408, 409, reflectedlight beam 1, 10-12, 14-18, 20, 21, 25, 422-424, 430-435 83, 84, 95, 97,98, 103 411, 412 proximity sensor light beams 38, 39 420 focus 39 425extension of light beam 82 501 proximity sensor bar 10-23, 82, 90-93,96-108 510, 511 proximity sensor bar 27-31, 36-42, 44, 47, 49-51, 53,83-88 512, 513 wireless communications chip 89, 91-93 601, 602 arrow 14,15 603-623 location 41, 42, 44, 53 630, 631 wireless communication 90-92channel 701 processor 10-18, 20, 21, 23, 91-93 702 applicationsprocessor 91 703 server 82 801, 802 object 10-13, 18, 20, 21, 23 803motor 14, 15 804, 805 shaft 14, 15 806, 807 reflective object 14-17 810,811 hand 27-29, 32-34, 44 812 bracelet 32-34 813 mirror 35-39 814reflection 37 815 mounting bracket 35, 36 816 display screen 35, 38-40,83, 84, 86-88 817-819 finger 38, 39, 44, 45, 96-98 820 projector 40 821object 41, 42, 49-51, 53 822 cup 43 823 debugger 47 824 socket 47 825,826 object 83, 84 827-829 folding mirror 101, 104-106 830 top cover 87,88 831 bottom cover 87, 88 832 tv 90 833 mobile phone 82, 90 834Bluetooth dongle 90 835 object location 95 836 curved display 94, 95 837flexible display 96-98 838 retractable display 100  839 retractabledisplay housing 99, 100 840 slot 99, 100 841 rotating stand 82 842mirror display 82 843 person 82 844 avatar 82 850, 851 key 105, 106 852,856-861 rods 105, 106, 108 853 springs 105  854, 855 block 105, 106 901,902 GUI 90 910-916, 916″, 919, 919′, 926, hotspot location 2-5, 10-13,16-18, 20-23, 926″, 929, 929′, 930, 931, 936, 77, 78, 103 936″, 939,939′, 940-942, 944, 945, 961-969 970, 971 detection plane 27, 33, 34,40, 49-51, 101, 102, 104-108 972 QR code 30, 31 973 display screengraphic 31, 34, 37, 40 974 field of view 31, 32, 34 975 headset 30-32,34 976 semi-transparent display 30 977 camera 30 978 image processingunit 30 979 2d scan information 45 980, 981 3d scan information 46, 48982 detection plane 49-51 983 3d scan information 43 985, 986 objectdetection 50, 51 987 3d model of scanned object 72 989 partial objectcircumference 22 990 detection image 19, 22 991-995 ring of six hotspotlocations in a 25 hyperboloidNumbered elements in the 1000's are stages in a process flow.

DETAILED DESCRIPTION

Throughout this description, the terms “source” and “emitter” are usedto indicate the same light emitting elements, inter alia LEDs, VCSELsand lasers, and the terms “sensor” and “detector” are used to indicatethe same light detecting elements, inter alia photodiodes.

Reference is made to FIG. 1, which is a simplified illustration of lightbeing emitted from light sources along a solid line, and being reflectedalong dashed lines to light sensors, in accordance with an embodiment ofthe present invention. FIG. 1 shows a central beam 407, and reflectionpaths 408, 409 represented by the dashed lines to light sensors, inaccordance with an embodiment of the present invention. All of thedashed lines in FIG. 1 represent paths of reflected light that aremaximally detected as explained below. FIG. 1 shows how light is emittedin a collimated beam. Light that hits an object is reflected diffusely.Sensors detect maximum incoming light from reflections in two narrowcorridors that extend from the sensor in two fixed directions, e.g., 30°for all emitter light beams on one side of the sensor and −30° for theremaining emitter light beams on the other side of the sensor.

The amount of light that travels from one source to one sensor dependson how centered the reflected object is on the source's beam, and howcentered it is on the sensor's detection corridor. Such a source/sensorpair is referred to as a “hotspot”. The location at which a reflectiveobject generates the highest amount of light for a source/sensor pair isreferred to as the “hotspot location” or the “target position” for thatsource/sensor pair. A proximity sensor according to the presentinvention measures the transmitted amount of light for eachsource/sensor pair, and each such measurement is referred to as a“hotspot signal value”. The measurement normalizes all hotspot signalvalues to have the same range.

Since light that hits an object is reflected diffusely and reflectionsare maximally detected in two narrow corridors at opposite sides of thelight beam, the present specification refers to a forward directiondetection based on all of the narrow detection corridors in a firstdirection, and a backward direction detection based on all of the narrowdetection corridors in the second direction. Stated differently, theforward direction includes all detections of source/sensor pairs inwhich the sensor of the pair has a higher location index than the sourceof the pair, and the backward direction includes all detections ofsource/sensor pairs in which the sensor of the pair has a lower locationindex than the source of the pair. The forward direction may be left orright, depending on device orientation. A hotspot where the sensor looksin the backward direction is referred to as a “backward hotspot”, andvice versa for those looking forward.

Reference is made to FIG. 2, which is an illustration of backward andforward hotspot signal values, i.e., signal values for source/sensorpairs, in accordance with an embodiment of the present invention.Hotspot signal values are sampled with an object placed at locations ina dense grid spanning all hotspot locations; i.e., all locations atwhich an object can be placed such that the source/sensor pairs willdetect a reflection value. FIG. 2 shows the maximum of all hotspotsignal values at object locations within a region that spans 3×3 hotspotlocations, or target positions, separately for backward and forwardhotspots. In FIGS. 2-5 hotspot locations are indicated as numberedelements 961-969 only in the illustrations of backward hotspot signalvalues. In FIGS. 2-5 the hotspot locations in the illustrations offorward hotspot signal values are not indicated as numbered elements inorder to avoid cluttering the figures.

Reference is made to FIGS. 3-5, which are illustrations showing a signalvalue relationship between neighboring hotspots within 3×3 grids offorward and backward hotspots, in accordance with an embodiment of thepresent invention. FIGS. 3-5 show relationships between two adjacenthotspot signal values. Each curve follows a fixed relationship value,similar to a topological map. FIG. 3 shows the signal value relationshipbetween top-middle and center hotspots. FIG. 4 shows the signal valuerelationship between right-middle and center hotspots. FIG. 5 shows thesignal value relationship between top-right and center backwardhotspots, and between top-middle and right-middle forward hotspots.

Reference is made to FIG. 6, which is an illustration showing arelationship between two signal values v0 and v1 (solid lines) expressedas r=log(v1)−log(v0) (dashed line), in accordance with an embodiment ofthe present invention. This relationship is drowned in noise when eitherof the signal values has a low signal-to-noise ratio (SNR).

The signal value relationship between two vertically adjacent hotspotscorresponds to a curve in FIG. 3. If the signal values are assumed to benormally distributed with a certain standard deviation, then thatassumption may be used to find an interpolated location between thehotspot locations according to FIG. 6, referred to as a “crossing”.Similarly, vertically adjacent hotspots create a second crossing. Therationale is that the object location is somewhere between the twocrossings. If the curves in FIG. 3 are all straight and parallel, thiswould be accurate. However, curvature causes inaccuracy.

In order to account for such curvature, the location between thecrossings is found using the same method, but from the relationships ofhorizontally adjacent hotspots. The curves are now those in FIG. 4.Instead of interpolating all hotspots to calculate the object'slocation, a shortcut is used. The vertical crossings are thought of asvirtual hotspots, and each virtual signal value is estimated based onthe real hotspot signal values and their respective distance from thecrossing. The virtual hotspots are interpolated to determine the objectlocation directly.

In some embodiments of the invention, an object is moved throughout thedetection area and signal values for all locations in the detection areaare recorded and stored. In such cases, an object's current location isderived by matching the current source/sensor pair detection signalvalues with the stored recorded values and selecting the correspondingstored location in the detection area.

Reference is made to FIG. 7, which is an illustration showing use oftriangles to mark areas surrounding hotspots at which an objectgenerates strong signal values, in accordance with an embodiment of thepresent invention. The mapping from two-dimensional signal relationshipsto three-dimensional location and reflectivity is similar in alltriangles; especially so in triangles of the same orientation and in thesame horizontal band, i.e., being the same distance from the proximitysensor. This means that the mapping needs to be learned and stored foronly a few triangle groups. It may be observed in FIG. 2 that there aretriangular areas spanned by three hotspots, in which those three hotspotsignal values are all relatively high. Some of these are drawn in FIG.7. This means that the three pairwise relationships between thosesignals will be above noise within the area. Out of those threerelationships one is redundant, since it is derivable from the othertwo. Within such a triangle, two signal relationships map to a locationwithin that triangle. It also maps to the reflectivity of the objectrelative to the observed hotspot signal values. These triangular areascover the whole screen, so the location and reflectivity of an object isfound by finding the triangle that is spanned by the hotspots with thehighest signal values, and mapping the signal relationships to locationand reflectivity.

The mapping transform takes the vertical (FIG. 3) and diagonal (FIG. 5)signal relationships as input. The input 2D space, from minimum tomaximum observed in each dimension, is covered by a 9×9 grid of nodes.Each node contains a location expressed in a frame of reference spannedby the triangle's edges. The location may be slightly outside thetriangle. The transform contains a compensation factor that, whenmultiplied with the highest signal value, gives the reflectivity of theobject. The four nodes closest to the input are interpolated withbi-linear interpolation.

All hotspots that have a signal value above a certain threshold, andthat are stronger than all its eight immediate neighbors, are evaluatedfor possible object detections. All six triangles that use the maximumhotspot are screened as possible contributors to the detection. Eachtriangle is given a weight that is calculated as the product of all itshotspot signal values. The highest three are kept, and their weights arereduced by that of the fourth highest. The kept triangles are evaluated,and their results are consolidated to a weighted average, using theweights used for screening.

Using a robot to place a stylus at known locations opposite a proximitysensor of the present invention and recording the resulting detectionsignals, enables quantifying the accuracy of the algorithm. The recordedsample signal values are sent as input to the algorithm in random order,and the calculated detection locations based on these inputs arecompared to the actual sample locations.

Reference is made to FIG. 8, which is an illustration of detectionerrors across a 100×64 mm touch screen, in accordance with an embodimentof the present invention. The 2D error vector is coded according to thelegend at the right in FIG. 8. The legend circle radius is 5 mm. FIG. 8shows how large, and in what direction, the error is for samples acrossthe screen.

Reference is made to FIG. 9, which is an illustration of a 2D histogramof sample error vectors, in accordance with an embodiment of the presentinvention. The units of the axes are mm. FIG. 9 shows the distributionof the errors. TABLE I below provides the quantified accuracy values.

TABLE I Measurement Value Error distances 50:th percentile 0.43 mm Errordistances 95:th percentile 1.04 mm Error distances 99:th percentile 1.47mm True positives 100.0% False positives 0.0%

Reference is made to FIGS. 10-13, which are simplified illustrations ofa proximity sensor, in accordance with an embodiment of the presentinvention. Proximity sensor 501 includes light sources 101-110 and lightsensors 201-211, each light source being situated between two of thelight sensors. Proximity sensor 501 also includes a plurality of lenses,such as lenses 301-304, each lens being positioned in relation to tworespective neighboring ones of the light sensors such that lightentering that lens is maximally detected at a first of the two lightsensors when the light enters that lens at an acute angle of incidenceθ1, and light entering that lens is maximally detected at the other ofthe two light sensors when the light enters that lens at an obtuse angleof incidence θ2. The lens is positioned in relation to the light sourcesituated between these two light sensors such that the light from thelight source is collimated as it exits proximity sensor 501. Thisarrangement provides the two narrow corridors that extend from eachsensor in two fixed directions away from opposite sides of the projectedlight beams as discussed hereinabove with respect to FIG. 1.

FIG. 10 shows a forward reflection path of maximum detection for hotspot913 generated by source/sensor pair 104/207, whereby light from source104 reflected off object 801 is maximally detected by sensor 207, andFIG. 11 shows a backward reflection path of maximum detection forhotspot 914 generated by source/sensor pair 109/207, whereby light fromsource 109 reflected off object 801 is maximally detected by sensor 207.FIGS. 10 and 11 show how sensor 207 is situated with respect toneighboring lenses 303 and 304 such that sensor 207 receives maximumforward reflection values via lens 303, and maximum backward reflectionvalues via lens 304.

As explained above with respect to FIG. 1, the intersections outsideproximity sensor 501 between the projected light beams and the corridorsof maximum detection provide a map of hotspots. Four hotspots areillustrated in FIGS. 12 and 13, two of which are numbed 940 and 941. Anobject 801 is shown nearest hotspot 940 in FIG. 12. Thus, the maximumdetection of object 801 is generated by source/sensor pairs 104/202 and104/207. Source/sensor pair 104/202 provides backward detection, andsource/sensor pair 104/207 provides forward detection, as discussedabove. Additional detections are generated by other source/sensor pairs,e.g., forward detection source/sensor pair 104/208, because light beamsfrom source 104 are scattered, and a portion thereof arrives at sensor208, but the amount of light detected at sensor 208 is significantlyless than that generated by source/sensor pair 104/207, because thescattered light arriving at sensor 208 does not travel on the corridorof maximum detection.

FIG. 13 shows proximity sensor 501 of FIG. 12, but object 801 is moved adistance d to the right. In this case similar detection signals aregenerated by forward source/sensor pairs 104/207 and 105/208. Each ofthese detections will be less than the detection generated bysource/sensor pair 104/207 in FIG. 12 and greater than the detectiongenerated by source/sensor pair 105/208 in FIG. 12, as explained abovewith reference to FIGS. 3-7. The location of object 801 between hotspots940 and 941 is calculated by interpolating the amounts of light detectedby source/sensor pairs 104/207 and 105/208. As discussed above withreference to FIG. 7, a location of object 801 is calculated byperforming at least two interpolations between amounts of light detectedby source/sensor pairs that correspond to three neighboring hotspots,the neighboring hotspots being the vertices of a triangle in thedetection space.

In order to determine how to interpolate the detected amounts of light,detection sensitivities are calculated in the vicinities of the hotspotsusing a calibration tool that places a calibrating object having knownreflective properties at known locations in the detection zone outsideand adjacent to proximity sensor 501. At each known location, aplurality of source/sensor pairs is synchronously activated and amountsof light detected by neighboring activated sensors are measured.Repetitive patterns in relative amounts of light detected by theneighboring activated sensors as the object moves among the knownlocation are identified. These patterns are used to formulate detectionsensitivities of proximity sensor 501 in the vicinities of the hotspotswhich are used to determine how to interpolate the amounts of lightdetected in order to calculate the location of a proximal object.

Reference is made to FIGS. 14 and 15, which are simplified illustrationsof calibration tools for the proximity sensor of FIGS. 10-13, inaccordance with an embodiment of the present invention. FIG. 14 shows afirst calibration tool that includes motor 803, and shafts 804 and 805that move reflective calibration object 806 horizontally and verticallyin relation to proximity sensor bar 501, as indicated by arrows 601 and602. At each location at which object 806 is placed, a plurality ofsource/sensor pairs that correspond to hotspots in the vicinity of thatlocation are activated and the amounts of light detected are used todetermine the sensitivity in the vicinity of those hotspots. Multiplesuch source/sensor pairs that share a common light source are activatedsimultaneously.

In some embodiments, a calibration tool, either that illustrated in FIG.14 or that illustrated in FIG. 15, is used on a representative sample ofproximity sensor 501 units, and the interpolation methods derivedtherefrom are applied to other similar units. In other embodiments eachproximity sensor 501 is individually calibrated, in order to provideinterpolations tailored to that specific proximity sensor.

FIG. 15 shows a second calibration tool that differs from that of FIG.14 in the size and shape of the reflective calibration object. In FIG.14 calibration object 806 is modeled as a finger or stylus typicallyused with proximity sensor bar 501, whereas in FIG. 15 calibrationobject 807 is a rod that spans the length of proximity sensor bar 501.The rod is covered in a material having reflective properties similar tothose of skin or of a stylus typically used with proximity sensor bar501. In the calibration tool of FIG. 15, shaft 805 remains at a fixedlocation on shaft 804, such that object 807 only moves toward and awayfrom proximity sensor bar 501, as indicated by arrows 602. In this case,at each location of object 807 the light sources are activated one afterthe other and, during each light source activation, any of the lightsensors 201-211 that may reasonably be expected to detect a significantreflection therefrom are activated. In some embodiments, all of thelight sensors 201-211 are simultaneously activated with each lightsource activation.

In addition to determining interpolation methods, the calibration toolsare used to map the locations of the hotspots that correspond to thesource/sensor pairs. Often the locations of the hotspots are shiftedfrom their expected locations due to mechanical issues such as impreciseplacement or alignment of a light source or light detector withinproximity sensor 501. When used to this end, each proximity sensor unitneeds to be calibrated and the calibration tool of FIG. 15 is moreefficient than that of FIG. 14.

Reference is made to FIGS. 16 and 17, which are simplified illustrationsshowing that the calibration tool of FIG. 15 identifies how the emittersand detectors of the proximity sensor have been mounted therein, inaccordance with an embodiment of the present invention. FIGS. 16 and 17show how imprecise placement of a light sensor (FIG. 16) or a lightsource (FIG. 17) is identified by the calibration tool of FIG. 15. FIG.16 shows three rows of hotspots including hotspots 910-912, 919, 929,and 939. These are expected hotspot locations, i.e., proximity sensor501 is designed to provide maximum detections of reflected light forrespective activated source/sensor pairs when an object is placed atthese locations. This is verified as calibration rod 807 moves closer toproximity sensor 501. Each row of hotspots is situated at a fixeddistance from proximity sensor 501. Three distances are shown: H1, H2and H3.

FIG. 16 shows how, when light sensor 207 is placed slightly to the leftof its correct position within proximity sensor 501, maximum detectionmeasured at this light sensor corresponds to hotspot locations 919′,929′ and 939′. Calibration rod 807 enters these positions at differentdistances than those expected. FIG. 16 shows how calibration rod 807arrives at hotspot location 919′ when it is a distance H3′ fromproximity sensor 501. By analyzing a series of local maximum detectionsthat share a common light sensor and occur at different distances thanthose expected, the calibration system detects the offset of a lightsensor from its expected position. In some embodiments processor 701controls, or receives input from, motor 803 and processor 701 updatesits map of hotspots according to the actual local maximum detections.

FIG. 17 shows how, when light source 104 is placed slightly to the leftof its correct position within proximity sensor 501, maximum detectionmeasured for source/sensor pairs that include light source 104 areshifted from expected hotspot locations 916, 926 and 936, to positions916′, 926′ and 936′. FIG. 17 shows how calibration rod 807 arrives athotspot position 916′ when it is a distance H3′ from proximity sensor501. By analyzing a series of local maximum detections that share acommon light source and occur at different distances than thoseexpected, the calibration system detects the offset of the light sourcefrom its expected position.

A proximity sensor according to the present invention is used toestimate a partial circumference of a proximal object. Reference is madeto FIG. 18, which is a simplified illustration showing a proximitysensor detecting a proximal object, in accordance with an embodiment ofthe present invention. FIG. 18 shows proximity sensor 501 and proximalobject 802. Four hotspot locations 939-942 along the edge of object 802facing the proximity sensor are shown. The reflection values associatedwith these hotspot locations are used to estimate the contour of thisedge.

As described above, each hotspot location is associated with one forwardsource/sensor pair and one backward source/sensor pair. In FIG. 18,hotspot location 940 is associated with source/sensor pairs 104/202 and104/207.

The reflection values are used to generate a two-dimensional pixel imageof reflection values indicating where reflective surfaces arepositioned. For example, when all hotspot locations for allsource/sensor pairs in proximity sensor 501 are assigned theirrespective, normalized reflection values, the result is atwo-dimensional image. The reflection values in different embodimentsare normalized within a range determined by the number of bits providedfor each pixel in the two-dimensional image, e.g., 0-255 for 8-bit pixelvalues, and 0-1023 for 10-bit pixel values.

Reference is made to FIG. 19, which is a simplified illustration of atwo-dimensional image of detection values, in accordance with anembodiment of the present invention. FIG. 19 shows proximity sensor 501whose detection plane is directed downward and the resultingtwo-dimensional image 990 of reflection values generated by an objectsituated within that detection plane. The pixel values in image 990 are8-bit values.

Because both forward and backward source/sensor pairs correspond to eachhotspot location, the reflection value for that location in thetwo-dimensional image can be derived in different ways. Namely, theforward-direction source/sensor pair can be used, or thebackward-direction source/sensor pair can be used. In some embodiments,the average of these two values is used, and in other embodiments themaximum of these two values is used, such that some pixels derive theirvalues from forward-direction source/sensor pairs, and other pixelsderive their values from backward-direction source/sensor pairs.

Certain reflection values for source/sensor pairs are not caused by areflective object at the corresponding hotspot, but rather by strayreflections at entirely different locations. FIGS. 20 and 21 show howthese cases are identified. Once identified, the corresponding pixelvalues in the two-dimensional image are reset to zero.

Reference is made to FIGS. 20 and 21, which are simplified illustrationsshowing a detected reflection value for an emitter-receiver pair that isnot associated with that pair's hotspot location, in accordance with anembodiment of the present invention. FIG. 20 shows hotspot locations 940and 944 aligned along a common emitter beam path 401. Hotspot location940 corresponds to source/sensor pairs 104/202 and 104/207, and hotspotlocation 944 corresponds to source/sensor pairs 104/201 and 104/208. Itis clear from FIG. 20 that any light from emitter 104 is blocked byobject 802 well before it arrives at hotspot location 944, and thereforeany light detected at sensors 201 and 208 during activation of source104 is not generated by a reflective object at hotspot location 944, butis rather stray reflections off the object at other locations.Therefore, the reflection value appropriate for hotspot location 944 iszero.

This state is determined by the fact that source/sensor pair 104/202 hasa significant detected reflection value, indicating that a reflectiveobject is at corresponding location 940, and therefore, light beam 401does not arrive at location 944. Moreover, because the lenses and thesensors are configured such that the maximum detection arrives at thesensor when it is reflected at angle θ1 it is clear that thesource/sensor pair detecting the maximum reflection from among allsource/sensor pairs that share a common source is the pair detectingreflections from an object at, or near, the corresponding hotspotlocation. Indeed, in the example shown in FIG. 20 the detection valuefor source/sensor pair 104/202 is much greater than the detection valuefor source/sensor pair 104/201. For the same reason, the detection valuefor source/sensor pair 104/207 is much greater than the detection valuefor source/sensor pair 104/208. A similar situation is illustrated inFIG. 21, except that in this case the two hotspot locations are situatedalong a common detection path.

FIG. 21 shows hotspot locations 940 and 945 aligned along a commonmaximum detected reflection path 403. Hotspot location 940 correspondsto source/sensor pair 104/202, and hotspot location 945 corresponds tosource/sensor pair 105/202. It is clear from FIG. 21 that light fromonly one location can be reflected along path 403 onto receiver 202. Andbecause the detected reflection value for source/sensor pair 104/202 isgreater than the detection value for source/sensor pair 105/202, it issafe to assume that the reflecting object is at, or near, hotspotlocation 940, and the detection value for source/sensor pair 105/202 isnot caused by a reflective object at hotspot location 945. Therefore,the reflection value appropriate for hotspot location 945 is zero.

In general, an emitted light path LP, such as path 401 in FIG. 17, has aplurality of hotspot locations thereon, denoted P₁, P₂, . . . , P_(N),at different distances from proximity sensor 501, such as hotspotlocations 916, 926 and 936, in FIG. 17. When an object is located at oneof these locations, denoted P_(i) other hotspot locations P_(i+j) andP_(i−k) also have corresponding detection values. In such cases, thehotspot location P_(max) for which a maximum detection value is detectedfrom among hotspot locations along LP, is considered to correspond tothe object, and all detection values for hotpot locations further fromproximity sensor 501 are reset to zero. Detection values for hotpotlocations between P_(max) and proximity sensor 501 are retained. Often,two hotspot locations P_(max) and P_(max+1) are used to calculate thelocation of the object, as explained hereinabove, and in such casesP_(max+1) is not reset to zero.

Similarly, a reflected light path RP, such as path 402 in FIG. 16, has aplurality of hotspot locations thereon, denoted P₁, P₂, . . . , P_(N),at different distances from the proximity sensor 501, such as hotspotlocations 919, 929 and 939, in FIG. 16. When an object is located at oneof these locations, denoted P_(i) other hotspot locations P_(i+j) andP_(i−k) also have corresponding detection values. In such cases, thehotspot location P_(max) for which a maximum detection value is detectedfrom among hotspot locations along RP, is considered to correspond tothe object, and all detection values for hotpot locations further fromproximity sensor 501 are reset to zero. Detection values for hotpotlocations between P_(max) and proximity sensor 501 are retained. Often,two hotspot locations P_(max) and P_(max+1) are used to calculate thelocation of the object, as explained hereinabove, and in such casesP_(max+1) is not reset to zero.

In this manner, the two-dimensional pixel image is refined and begins torepresent the contour of the object facing the sensor. Reference is madeto FIG. 22, which is a simplified illustration of a detected partialcircumference of an object in a two-dimensional image of detectionvalues, in accordance with an embodiment of the present invention. FIG.22 shows the detected partial circumference 989 in the detection image990 of FIG. 19 and an example pixel 915 having a non-zero detectionvalue, but whose appropriate reflection value is zero, as explainedhereinabove.

The next step is to filter the pixels in this image to obtain sub-pixelprecision for the location of the object's contour between hotspotlocations. After calculating sub-pixel values, various edge detectionfilters are applied to the two-dimensional pixel image to identify theedges of the object facing the sensor and discard stray reflections.Known edge detection filters include Sobel, Canny, Prewitt, Laplace,gradient. This edge information is used to determine a length of thisportion of the object, i.e., a partial circumference of the object, andits location.

The length of the detected portion of the object is calculated usingdifferent methods, in accordance with different embodiments of theinvention. Some embodiments determine the number of pixels, orsub-pixels, along the detected portion of the object. Other embodimentscalculate the sum of the distances between each pair of neighboringpixels, or sub-pixels, along the detected portion of the object. Stillother embodiments determine an equation for a curve that passes througheach of the pixels, or sub-pixels, along the detected portion of theobject, and calculates the length of the partial circumference of theobject according to this equation.

In some embodiments, an estimate of the partial circumference iscalculated based on three points: the point on the object for whichthere is a maximum detection value and the two outermost points alongthe partial circumference.

Reference is made to FIG. 23, which is a simplified illustration of amethod of estimating a partial circumference of an object, in accordancewith an embodiment of the present invention. FIG. 23 shows point 940 forwhich there is a maximum detection value and two outermost points 939and 941 along the partial circumference of object 802. An estimate ofthe partial circumference is the sum of the distances from point 939 topoint 940 and from point 941 to point 940. In order to further refinethis calculation the system calculates the sub-pixel coordinates ofthese three positions using the immediate neighbors of the respectivehotspot locations 939-941, but does not calculate sub-pixel locationsfor any other pixels in the two-dimensional pixel image. Point 940, or arespective sub-pixel location, for which there is a maximum detectionvalue is used as the object's coordinates.

In other embodiments of the invention, the shape of the proximity sensoris not a straight line, but circular, or wave-shaped to provide a 3Ddetection volume, instead of a 2D detection plane. In such alternativeembodiments, the emitters and receivers are still alternated as they arein proximity sensor 501, and each emitter is paired with each of thereceivers as a source/sensor pair having a corresponding hotspot withina 3D volume above the proximity sensor.

Reference is made to FIG. 24, which is an illustration of a saddle roofor hyperbolic paraboloid corresponding to the emitter light paths andreflected light paths of a 3D proximity sensor, in accordance with anembodiment of the present invention.

Reference is made to FIG. 25, which a simplified illustration of acircular arrangement of six emitters and six receivers arrangedalternatingly along a circular base that provides 30 hotpot locationsalong a 3D hyperboloid, in accordance with an embodiment of the presentinvention. FIG. 25 shows emitters 101 and 102, and receivers 201 and202, which, together with the remaining emitters and receivers, provide30 hotpot locations along a 3D hyperboloid, in accordance with anembodiment of the present invention. FIG. 25 shows five rings, 991-995,of hotspot locations along the height of the hyperboloid.

Reference is made to FIG. 26, which is a simplified illustration of agrid representing emitted and reflected light beams from a circulararrangement of 16 emitters and 16 receivers arranged alternatingly alonga circular base, which provides 176 hotpot locations along a 3Dhyperboloid, in accordance with an embodiment of the present invention.Alternative configurations include 4 emitters and 4 receivers for astripped down hyperboloid, 3 emitters and 3 receivers for a regularoctahedron and 2 emitters and 2 receivers for a regular tetrahedron.These 3D proximity sensors are used inter alia for detecting in-air handwave gestures as the hand performing such a gesture passes through thehyperboloid.

Proximity sensors according to the present invention have numerousapplications for touch screens, control panels and new user interfacesurfaces. The proximity sensor can be mounted, e.g., on a wall, on awindow, or placed on a notebook, and it will provide touch and gesturedetection upon that item. These detected gestures are then used as inputto electronic systems. For example, a gesture along a wall is used todim the lighting in the room by mounting the sensor along an edge of thewall and communicating the detected gestures to the lighting system.Significantly, the proximity sensor is only mounted along one edge ofthe detection area, reducing component cost and providing moreflexibility for industrial design of touch screens and touch sensitivecontrol panels.

Reference is made to FIGS. 27-29, which are simplified illustrations ofa first input accessory 510 for virtual reality (VR) systems, inaccordance with an embodiment of the present invention. The purpose ofthis input accessory 510 is to detect grasping and flexing gesturesperformed by hand 810. The accessory is the proximity sensor describedhereinabove with reference to FIGS. 10-13 and accommodated in a housingthat fits in a user's hand 810. As explained hereinabove, this proximitysensor includes a printed circuit board (PCB) on which an array ofinterleaved individually activatable laser diode emitters and photodiodedetectors is mounted, wherein the laser diode emitters emit light alonga detection plane 971 outside of the housing, and the photodiodedetectors generate detection signals when the light emitted by the laserdiode emitters is reflected by an object that passes through thedetection plane back to the photodiode detectors.

Input accessory 510 detects grasping and flexing gestures by having theuser hold input accessory 510 in a manner that its detection plane 971extends above and across the prone fingers of hand 810, as illustratedin FIG. 27, whereas flexed fingers of hand 810 enter detection plane971, as illustrated by FIGS. 28 and 29. The processor in accessory 510identifies which fingers are flexed by mapping fingers on hand 810 tolocations along the length of accessory 510. The processor is incommunication with a VR system to report detected grasping and flexinggestures.

The processor is connected to the laser diode emitters and photodiodedetectors, to (i) selectively activate the laser diode emitters andphotodiode detectors, (ii) detect a plurality of grasping and flexinggestures performed when holding the accessory in the palm of hand 810 asone of more of the hand's fingers pass through the projection plane,based on detection signals generated by the photodiode detectors, and(iii) transmit input signals to the VR system, corresponding to thedetected grasping and flexing gestures. FIG. 28 shows a first detectedgesture in which two fingers are flexed and two others are prone, andFIG. 29 shows a second detected gesture in which four fingers areflexed. In response to these detected gestures the VR system performsactions in its virtual environment. For example, in response to adetected grasping gesture the VR system simulates the user picking up anobject in the virtual environment, and in response to detecting arelease gesture of stretching out one's fingers the VR system simulatesthe user releasing a held object in the virtual environment.Combinations of gestures by different fingers of hand 810 activatedifferent functions. E.g., bending the middle finger and the indexfinger activates a first function, and bending the ring finger and theindex finger activates a second function.

Reference is made to FIGS. 30 and 31, which are simplified illustrationsof a second input accessory 510 for VR systems, in accordance with anembodiment of the present invention. The purpose of this input accessory510 is to provide a virtual keyboard or trackpad. The accessory is theproximity sensor described hereinabove with reference to FIGS. 10-13accommodated in a housing on which QR code 972, or another identifiablegraphic, is provided. The VR system includes headset 975 having asemi-transparent display 976 operable to overlay graphics, such asscreen graphic 973, on the wearer's field of view 974. Headset 975 alsoincludes a camera 977 that captures at least a portion of the wearer'sfield of view 974, and an image processing unit 978 that extractsinformation from the captured images. Image processing unit 978 is shownoutside headset 975 in FIG. 30 for clarity. However, image processingunit 978 may be mounted in headset 975 or outside it.

When camera 977 captures an image of input accessory 510, imageprocessing unit 978 extracts information, namely, (i) an identifier,from QR code 972, identifying the type of keyboard or trackpad to berendered on display 976, and (ii) the location of QR code 972 in thewearer's field of view 974. Display 976 renders the thus identifiedkeyboard anchored at or near the QR code location in the wearer's fieldof view 974 such that gestures performed on the virtual keyboard ortrackpad are inside detection plane 971 of accessory 510, and thevirtual keys are seen by the wearer as being actuated. Accessory 510also includes communication circuitry that sends the detected gestureinput to headset 975 or to another unit of the VR system.

Reference is made to FIGS. 32-34, which are simplified illustrations ofa third input accessory for VR systems, in accordance with an embodimentof the present invention. FIGS. 32-34 show a bracelet 812 for use in aVR system similar to that shown in FIGS. 30 and 31. FIGS. 32-34 showbracelet 812 worn on a user's arm above hand 810. FIG. 32 shows an armwearing bracelet 812 as seen by the naked eye. FIG. 33 shows a detectionzone 971 on the user's arm provided by a light-based proximity sensorhoused in bracelet 812, similar to that described hereinabove withreference to FIGS. 10-13. FIG. 34 shows the user's arm as seen by thewearer of headset 975. A QR code or other identifiable graphic (notshown) on bracelet 812 is visible to camera 977 in headset 975, and theidentification of that code or graphic by image processing unit 978causes headset display 976 to render a corresponding keyboard 973anchored at or near the QR code location, on the user's arm. Gesturesperformed on the virtual keyboard are inside detection plane 971 ofbracelet 812, and the virtual keys are seen by the wearer as beingactuated by the user's other hand 811. Bracelet 812 also includeswireless communication circuitry that sends the detected gesture inputto headset 975.

Reference is made to FIGS. 35-37, which are simplified illustrations ofan interactive mirror, in accordance with an embodiment of the presentinvention. FIG. 35 is a view from behind the mirror, showing one-waymirror 813, display 816 and bracket 815 for maintaining the mirrorupright or mounting it on a wall. The front of display 816, on whichimages are rendered, faces one-way mirror 813 that so that imagesrendered on display 816 are visible to a user facing the front of mirror813.

FIG. 36 shows a front-facing view of one-way mirror 813. Proximitysensor 510, described hereinabove with reference to FIGS. 10-13, issituated along an edge of mirror 813 and its detection plane extendsalong the upper surface of mirror 813 to detect user gestures thatinteract with the underlying graphic on display 816. Proximity sensor510 is in communication with display 816, so that inputs detected bysensor 510 are transmitted to display 816. FIG. 37 shows both areflection 814 and an image from display 816, seen in one-way mirror813.

Reference is made to FIGS. 38 and 39, which are simplified illustrationsof an interactive mirror with an optical sensor mounted behind themirror, in accordance with an embodiment of the present invention. FIG.38 is a profile view of an alternative configuration for an interactivemirror, in which optical proximity sensor 510 is mounted behind themirror. FIG. 38 shows one-way mirror 813, display 816 and proximitysensor 510 mounted behind mirror 813, and a pair of reflectors 305 forredirecting proximity sensor light beams 411 in front of mirror 813 andredirecting light beams 412 reflected by finger 817 back to proximitysensor 510.

FIG. 39 is a profile view of a lens for redirecting light beams frombehind mirror 813 to the front of mirror 813 and back, to replacereflectors 305. FIG. 39 shows one-way mirror 813, display 816 andproximity sensor 510 mounted behind mirror 813, and lenses 306 forredirecting proximity sensor light beams 411 in front of mirror 813 andredirecting light beams 412 reflected by finger 817 back to proximitysensor 510. Each lens 306 includes an outer refractive surface 307 andan inner reflective surface 308 that have a shared focus 420. Lenses 306are described in U.S. Pat. No. 8,587,562, entitled LIGHT-BASED TOUCHSCREEN USING ELLIPTICAL AND PARABOLIC REFLECTORS, incorporated herein byreference in its entirety.

Reference is made to FIG. 40, which is a simplified illustration of aninteractive mid-air display system, in accordance with an embodiment ofthe present invention. FIG. 40 shows an interactive mid-air displayfeaturing display 816 and a projector 820 that projects the image ondisplay 816 in-air above the display where it is seen as image 973. Theinteractive mid-air display includes proximity sensor bar 510 directedalong the plane of image 973 for mapping input gestures onto projectedimage 973 and sending the detected gestures back to display 816.

Proximity sensors according to the present invention are used togenerate 3D information about one or more proximal objects by repeatedlymoving the proximity sensor, such that light emitted by the emitters isprojected into a different planar airspace after each move, repeatedlyselectively activating the emitters and the photodiode detectors,repeatedly identifying a shape of the object in the planar airspace,based on outputs of the photodiode detectors and the known angles ofmaximum emitted light intensity and maximum detected reflections, andcombining the identified shapes in the different planar airspaces togenerate a three-dimensional image of the object, based on orientationsof the planar airspaces. The generated 3D information identifies theobject's shape, a 3D volume in which the object is contained, and theobject's location.

Reference is made to FIG. 41, which is a simplified illustration showingan upright proximity sensor bar 510 being repeatedly moved in relationto a detected object, to generate a three-dimensional image of theobject, in accordance with an embodiment of the present invention. FIG.41 shows object 821 being repeatedly scanned by proximity sensor bar 510at locations 603-606 along a first edge of object 821, and at locations607-611 along a second edge of object 821. At each scan locationproximity sensor bar 510 generates an image of object 821 in theproximity sensor's 2D detection plane. A 3D image of the object isgenerated by mapping the different 2D detection planes and theirrespective detections to 3D space. In some embodiments of the invention,proximity sensor bar 510 includes an inertial measurement unit (IMU)that tracks the position and orientation of the sensor bar, and itsdetection plane, to facilitate mapping the different 2D detection planesand their respective detections to a common 3D space. The IMU includesone or more components operable to measure and report the sensor bar'sspecific force and angular rate, inter alia, accelerometers andgyroscopes.

Reference is made to FIG. 42, which is a simplified illustration showinga horizontal proximity sensor bar being repeatedly moved in relation toa detected object, to generate a three-dimensional image of the object,in accordance with an embodiment of the present invention. FIG. 42 showsobject 821 being repeatedly scanned as proximity sensor bar 510 movesupward at locations 612-615 along a first edge of object 821, and asproximity sensor bar 510 moves downward along a second edge of object821. At each scan location proximity sensor bar 510 generates an imageof object 821 in the proximity sensor's 2D detection plane. A 3D imageof the object is generated by mapping the different 2D detection planesand their respective detections to 3D space. Although FIGS. 41 and 42show only two edges of object 821 being scanned, additional edges arescanned as well, and their information is added to the 3D information.In some embodiments of the invention, for each 2D detection planeproximity sensor bar 510 detects multiple points on the outer edge ofobject 821 and those locations are combined to create a point cloud inthe 3D space. In other embodiments of the invention, for each 2Ddetection plane proximity sensor bar 510 identifies a contiguous edge ofobject 821 and those edges are combined to create a 3D model in the 3Dspace.

Reference is made to FIG. 43, which is an illustration of a scanned cupand scan data combined into a 3D model, in accordance with an embodimentof the present invention. FIG. 43 shows a cup 822 and 3D image 983. 3Dimage 983 was generated by scanning cup 822 with a proximity sensor bar(not shown) at a range of different heights and around the cup'scircumference and combining the scan information.

Reference is made to FIG. 44, which is a simplified illustration showingthree fingers being scanned by a proximity sensor on four differentsides of the fingers, in accordance with an embodiment of the presentinvention. FIG. 44 shows three fingers 817-819 of a hand 811 beingscanned by proximity sensor bar 510 at four locations 616-619 around thethree fingers.

Reference is made to FIG. 45, which is a simplified illustration showingdetections of the scan operations illustrated in FIG. 44, in accordancewith an embodiment of the present invention. FIG. 45 shows scan signals979 mapped to locations in the detection plane from one of the proximitysensor locations in FIG. 44. The locations of fingers 817-819 areindicated in signals 979.

Reference is made to FIG. 46, which is a simplified illustration showingsignal intensities in the scan operations illustrated in FIG. 44, inaccordance with an embodiment of the present invention. FIG. 46 shows a3D model 980 of the scan signals 979 of FIG. 45 mapped to locations inthe detection plane.

Reference is made to FIG. 47, which is a simplified illustration of adebugger interface unit scanned by a proximity sensor on four differentsides of the unit, in accordance with an embodiment of the presentinvention. FIG. 47 shows a debugger unit 823 being scanned by proximitysensor bar 510 at four sides of the unit. Scanning is performed atmultiple heights opposite each side of unit 823 in order to generate a3D image of the unit.

Reference is made to FIG. 48, which is a simplified illustration showingsignal intensities in the scan operations illustrated in FIG. 47, inaccordance with an embodiment of the present invention. FIG. 48 shows a3D image 981 of unit 823 created by combining scan information from themultiple scans discussed in relation to FIG. 47. The indent at the rightedge of 3D image 981 corresponds to socket 824 shown in FIG. 47.

As proximity sensor bar 510 is translated along the height of a scannedobject, three types of detection event are generated: NULL, OBJECT andSHADOW. A NULL event is generated when noise or no signal is detected,indicating that no object is situated in the proximity sensor detectionplane. An OBJECT event is generated when a high reflection is detected,indicating that an object is situated in the proximity sensor detectionplane. A SHADOW event is on the border between a NULL event and anOBJECT event, i.e., when some reflection is detected, but it is unclearif that reflection should be treated as noise or as an objectreflection. SHADOW events are generated, inter alia, when proximitysensor light beams pass near an object, for example when the detectionplane is just beyond the top or bottom of the object.

Reference is made to FIG. 49, which is a simplified illustration showinga proximity sensor directed above an object, and the resulting scaninformation, in accordance with an embodiment of the present invention.FIG. 49 shows sensor 510 at a height h1, whose detection plane 971 isabove object 821. Detections of reflected light are mapped to detectionplane 982. In this case there are no object reflections and alldetections are NULL events, as detection plane 971 is above object 821.

Reference is made to FIG. 50, which is a simplified illustration showinga proximity sensor directed at an upper edge of an object, and theresulting scan information, in accordance with an embodiment of thepresent invention. FIG. 50 shows sensor 510 at a height h2, whosedetection plane 971 coincides with the top of object 821. Detections 985of reflected light are mapped to detection plane 982. In this case, thedetections are all SHADOW events.

Reference is made to FIG. 51, which is a simplified illustration showinga proximity sensor directed at the center of an object, and theresulting scan information, in accordance with an embodiment of thepresent invention. FIG. 51 shows sensor 510 at a height h3, whosedetection plane 971 intersects the middle of object 821. Detections 986of reflected light are mapped to detection plane 982. In this case, thedetections are OBJECT events.

Reference is made to FIG. 52, which is a graph of detection signals forindividual emitter-detector pairs generated by 11 scan sequencesperformed by a proximity sensor situated at a fixed location in relationto an object, in accordance with an embodiment of the present invention.FIG. 52 shows results of 11 scan sequences by a proximity sensor barplaced a fixed distance from an object. The x-axis represents an indexof the emitters in the proximity sensor bar, and the y-axis representssignal strength. For each emitter the maximum detected reflection isplotted on the graph. Although maximum signals for neighboring emittersdiffer, the maximum signal from each emitter remains stable over all 11activations, indicating that the signals are stable.

FIGS. 53 and 54 show how a distance between the sensor bar and adetected object is derived from the strength of the detector signal.Reference is made to FIG. 53, which is a simplified illustration showinga proximity sensor scanning an object at four different distances fromthe object, in accordance with an embodiment of the present invention.FIG. 53 shows sensor bar 510 scanning object 821 at four locations620-623, each a different distance from object 821. The magnitude of thedetected reflections of object 821 is weaker as the distance increases.Reference is made to FIG. 54, which is a radar plot of detection signalsgenerated by scan sequences performed at multiple locations around anobject, and at different distances from the object, in accordance withan embodiment of the present invention. FIG. 54 shows a radar plot ofdetection signals generated by the proximity sensor bar at 14 differentlocations, 40 mm-105 mm from the object with a 5 mm step betweenlocations. FIG. 54 shows that radar plots for detections generated atlocations 20 mm apart do not intersect. Thus, detection signal strengthis used to detect a distance between the object and the sensor bar at aresolution of 20 mm.

Reference is made to FIG. 55, which is a 3D graph of signal intensitiesgenerated by a proximity sensor bar placed along four sides of a scannedobject at a first height, in accordance with an embodiment of thepresent invention. Reference is made to FIG. 56, which is a 3D graph ofsignal intensities generated by the proximity sensor bar placed alongthe four sides of the scanned object at a second height, in accordancewith an embodiment of the present invention. Reference is made to FIG.57, which is a 3D graph of signal intensities generated by the proximitysensor bar placed along the four sides of the scanned object at a thirdheight, in accordance with an embodiment of the present invention.Reference is made to FIG. 58, which is a 3D graph of signal intensitiesgenerated by the proximity sensor bar placed along the four sides of thescanned object at a fourth height, in accordance with an embodiment ofthe present invention. The scanned object is debugger unit 823 shown inFIG. 47. Reference is made to FIG. 59, which is a simplifiedillustration of a 3D image of a scanned object, generated by combiningdetections of a proximity sensor placed at locations around the object,and at different heights in relation to the object, in accordance withan embodiment of the present invention. FIG. 59 shows a 3D image of thescanned object, created by combining the four sets of scan informationshown in FIGS. 55-58.

Reference is made to FIGS. 60-63, which are illustrations showingtranslating the location of an object, as detected in a single proximitysensor's detection plane, to a common detection plane formed by theunion of multiple proximity sensor detection planes, in accordance withan embodiment of the present invention. FIGS. 60-63 show how to combinescan information generated along different sides of an object. Each ofFIGS. 60-63 shows the object detection in a 19×19 pixel scan area,generated by a proximity sensor bar placed along one side of the object.Black squares indicate the location or position of the object within thescan area as detected by the proximity sensor bar, irrespective of theother scans, whereas the red squares indicate where the object islocated when all four scan areas are aligned to a single, 2D area. Thesetranslations are needed when the 19×19 pixel scan areas for each scanare not aligned with one another. Although FIGS. 60-63 show a 19×19pixel scan area, the size of the scan area depends on several factors,inter alia, the number of emitters, and the distance between emitters,on the proximity sensor bar.

Reference is made to FIG. 64 which is a simplified chart of a processfor converting scan information into a 3D representation of a scannedobject, in accordance with an embodiment of the present invention. Steps1001-1008 convert detection signals from a proximity sensor bar to a 3Dimage of a scanned object. Step 1001 (“Noise Filtering”) showsprocessing done at the signal level, e.g., spike removal and noiseremoval. Step 1002 (“2D Image Construction and Analysis”) is where thesignals from one scan of the proximity sensor are transformed into the2D domain. This typically involves the steps of applying a gain factoror otherwise normalizing the signals (“gain factor”), indexing each scanoutput according to the location of the proximity sensor during the scanfor the later step of combining multiple scans into a 2D frame (“scanno.”), removing signals associated with respective hotspots where thesignals were not generated by a reflective object at the hotspotlocation (“reflection detector”), edge detection (“image processing”),calculating 2D coordinates of the object's edges in the detection plane(“coordinate calculation”) and extracting various object properties,inter alia, object size, object coordinates in the proximity sensor scanplane and which of the proximity sensor emitter-detector pairs detectthe object.

At step 1003 (“Frame Design”) multiple 2D images of step 1002 capturedwhen the proximity sensor was placed at different locations around theobject, but along the same 2D plane, are selected and indexed tofacilitate combining them into a 2D image referred to as a “frame”.Thus, a frame represents a slice of the object along a 2D plane.Different frames represent parallel slices of the object at differentheights that can be combined into a 3D image of the object. Whenconstructing a frame, it may become apparent that certain ones of thescans are problematic. For example, the reconstructed frame of a solidobject should have a closed shape, but in one of the scans, the detectededge doesn't span the gap indicated by the neighboring scans on eitherside of that scan. In these cases, at step 1004 (“Frame NoiseFiltering”) the problematic scan information is either removed from theframe construction or the data in the problematic scan is filtered. Alsoat step 1004, noise arising from combining 2D information from differentscans into a coherent frame is filtered out (“Noisy frame filtering”).

At step 1005 multiple frames are combined to create a 3D image of theobject (“3D Image Construction”) as discussed hereinabove. This involvesthe steps of identifying the center of gravity of the object in eachframe (“center of gravity”) and using the center of gravity in adjacentframes to align those frames (“3D object alignment”). When the distancebetween the centers of gravity in adjacent frames is within a definedtolerance level, e.g., 5 mm, the frames are stacked with their centersof gravity aligned one above the other. When the distance between thecenters of gravity in adjacent frames is above the defined tolerance, itis understood that the object is not perpendicular to the 2D planes ofthe frames and the adjacent frames are stacked accordingly. In certaincases, pixels from some reconstructed frames extend beyond the 3Ddomain. In such cases, these outlying pixels are translated to thenearest edge of the 3D domain (“Object transition”), as explainedhereinbelow with reference to FIGS. 71-73.

At step 1006 the combined 3D image is filtered to smooth anydiscontinuities resulting from combining the different frames (“3DObject Continuity”). In some embodiments of the invention this is doneby translating object pixels at both sides of a discontinuity so thatthey represent a single common location on the object. This reduces thesize of the object.

At step 1007 lighting effects are added to the constructed 3D image forbetter visualization (“3D Object Visualization”). At step 1008, thesystem creates a 3D image from the combined 2D images. This 3D imageuses the z-axis either to plot the height of the object (“length view”),or alternatively, to indicate the intensity of the detected signals ateach mapped 2D coordinate (“signal view”).

Reference is made to FIGS. 65-70, which are illustrations of a scannedobject and a 3D image of the object generated from proximity sensordetections, in accordance with an embodiment of the present invention.FIG. 65 shows a scanned debugger unit on the left and the 3D imagegenerated from the multiple scans thereof on the right.

FIG. 66 shows a scanned ball on the left and the 3D image generated fromthe multiple scans thereof on the right.

FIG. 67 shows a scanned arrangement of four prosthetic fingers on theleft and the 3D image generated from the multiple scans thereof on theright. In the illustrated case of four fingers having different heightsarranged in a row, the height of the object detected when the proximitysensor scans that side of the arrangement in which most of the fingersare in the shadow of another, only one height is detected, namely, theheight of the tallest finger. When reconstructing the 3D image thisheight is used for all edges of the image. Therefore, in the 3D image onthe right of FIG. 67 all four fingers have the same height. In someembodiments, although a single height is used for the entirereconstructed object, the height used is the average of the two tallestheights detected during the scanning.

FIG. 68 shows a scanned prosthetic finger on the left and the 3D imagegenerated from the multiple scans thereof on the right.

FIG. 69 shows a scanned upright, hexagon-shaped coaster on the left andthe 3D image generated from the multiple scans thereof on the right.

FIG. 70 shows a scanned roll of adhesive tape on the left and the 3Dimage generated from the multiple scans on the right.

In certain embodiments of the invention, only a limited amount ofmemory, e.g., a screen buffer, is available to store the entirereconstructed 3D model of the scanned object. During reconstruction, ifa portion of the object juts out beyond the 3D domain, it is transferredto the nearest edge of the 3D domain. The following example clarifiesthis case.

Suppose the scanned object is shaped as an upside-down “L”. Theproximity sensor scans this object from all four sides at differentheights from the bottom up. Each set of four scans at a single height iscombined into a 2D slice of the object at the given height. These slicesare combined from bottom to top, with the first slices of the objectbeing mapped to the 3D domain. When the upper slices of the object areadded to the model, outermost pixels on the roof of the upside-down “L”are outside the 3D domain. These pixels are translated to the nearestedge of the 3D domain. This is a dynamic process that is performed whileadding the different 2D scans into a 3D model.

Reference is made to FIG. 71, which is a simplified flow chart of aprocess for converting scan information into a 3D representation of ascanned object, in accordance with an embodiment of the presentinvention. The process of FIG. 71 combines multiple 2D scan informationinto a 3D model of the object. At step 1011 all scan information fromall scans made is analyzed to identify object edges in the scaninformation; determine the size of the object; determine an initiallaunching point of the object in the 3D domain, based on the object'ssize; and, in each scan, identify the active emitter-detector pairs thatdetected the object. This last parameter is used to determine if theproximity sensor bar was shifted during scanning. At step 1012 thedetection signals are converted into 2D coordinates. At step 1013 aslice of the object is created by combining 2D scans taken by theproximity sensor from four sides of the object at a common height. Thisslice is also called a “frame”. At this stage the system connects theedges identified in the different 2D scans to form a closed shape, andalso calculates the center of gravity of that shape.

At step 1014 the process combines the frames of step 1013 into a 3Dobject. This process involves copying each successive frame into apre-allocated memory beginning at the initial launching point calculatedat step 1011 and ensuring that the object edges in neighboring framesare connected. In some embodiments, the degree of alignment between thecenter of gravity of each newly added frame and the center of gravity ofthe current 3D object is measured. Misalignment up to a threshold isassumed to arise from an actual variation in the contour of the object,but a large misalignment is assumed to arise due to errors that occurredwhen the scans were performed. In this latter case, the new frame istranslated so that its center of gravity is aligned with the center ofgravity of the 3D object. At step 1014 the system checks if, afteradding each new frame to the 3D object, any of the pixels in the newframe extend outside the 3D domain. If they do, those pixels are movedto the nearest edge of the 3D domain, thereby essentially shrinking theobject. At step 1014 the system converts the 2D coordinates of theobject to 3D coordinates.

Reference is made to FIG. 72, which is an illustration showingtranslating an object's bounding box within a shared detection plane, inaccordance with an embodiment of the present invention. FIG. 72 shows 3Dmodel 987 of a scanned elbow-shaped object. FIG. 72 shows a first centerof gravity 1 of the object's bounding box when only some of the lowerframes of the object have been added to the 3D model. FIG. 72 also showsa second center of gravity 2 of the object's bounding box when theentire object is included in the 3D model.

Reference is made to FIG. 73, which is a flow chart for a method ofgenerating a 3D image of a scanned object, by combining detections of aproximity sensor placed at locations around the object, and at differentheights in relation to the object, in accordance with an embodiment ofthe present invention. The flowchart shows how scan information fromscans of the proximity sensor at different locations are combined into aframe. The process begins after multiple scans of the object have beenperformed, each scan being stored and indexed based on the proximitysensor location with respect to the object during the scan. At step 1021one set of scan information is selected. At step 1022, the 2Dcoordinates of the object are read from the selected scan information,and at step 1023 an outermost coordinate is selected as a startingpoint. The series of concatenated edge coordinates is read from theselected scan information, and each coordinate is transformed, at step1025, to a common 3D space based on a transition model (step 1024). Thiscontinues until the end of the object edge in the selected scaninformation is reached. The process loops back to step 1021 to selectthe next set of scan information, namely, scan information related tothe object edge adjacent to the final coordinate from the previous scan.At step 1023, an outermost first pixel in this new set of scaninformation is mapped onto the same location as the final pixel of theprevious scan information. After all four edge scans have been processedin this way the system calculates the center of gravity of the objectslice in the 2D frame and compares it to the centers of gravity of theobject's previous frames. The expectation is that for an upright objectthese centers of gravity are aligned, and for a tilted or curved object,these centers of gravity are not aligned. In some embodiments, when thecenters of gravity of multiple consecutive frames are non-aligned, thesystem calculates a slope of the object based on the series ofnon-aligned frames and translates the coordinates of those framesaccording to that slope, on the assumption that the object is tilted orcurved as indicated by the non-aligned centers of gravity. When within afew consecutive frames the centers of gravity do not indicate aconsistent slope, e.g., the center of gravity of frames N+1 and N+2 isnot aligned with that of frame N, and the center of gravity of frame N+3is aligned with that of frame N, the method translates the coordinatesof frames N+1 and N+2 to align their centers of gravity with those offrames N and N+3, under the assumption that the non-alignment resultedfrom a lack of precision in placement of the sensor bar during scanning.

Next, the system retrieves the scan information for the next frame,namely, four scans taken at an incrementally higher height than theprevious scan. This process repeats until all of the scan informationhas been transformed to the shared 3D domain.

Reference is made to FIGS. 74 and 75, which are illustrations of scannedobjects and 3D images of those objects generated from proximity sensordetections, in accordance with an embodiment of the present invention.FIGS. 74 and 75 each show a scanned tapered object on the left and the3D image generated from the multiple scans on the right.

Reference is made to FIG. 76, which is an illustration of a scannedscotch tape dispenser and the heights of that dispenser detected on foursides thereof, in accordance with an embodiment of the presentinvention. FIG. 76 shows a tape dispenser that is scanned by a proximitysensor placed at locations around the dispenser's perimeter at multipleheights. The graph on the right shows four lines, each representingdifferent heights at which the object is detected along one edge of thetape dispenser. The height of the dispenser detected along the two longedges—front and rear—of the dispenser is greater than the heightdetected along the narrow—left and right—edges of the dispenser. This isdue to tapering at the left and right edges of the dispenser, which canbe seen in the figure.

As discussed hereinabove with respect to FIG. 23, sub-pixel calculationincreases the resolution of a detected object's size and position.Reference is made to FIGS. 77 and 78, which are illustrations of signalbehavior as a fingertip moves between hotspots, in accordance with anembodiment of the present invention. FIG. 77 shows signal behavior whenan object moves away from a hotspot, and FIG. 78 shows signal behaviorwhen an object moves towards a hotspot. FIGS. 77 and 78 show threeneighboring hotspots 961-963 between which the fingertip moves. Thefingertip spans two hotspot locations. At the beginning of the movementtracked in FIGS. 77 and 78 the fingertip is fully detected at hotspots961 and 962 and is not detected at all at hotspot 963; at the end of thetracked movement the fingertip is fully detected at hotspots 962 and 963and is not detected at all at hotspot 961. FIG. 77 is a graph thattracks the differences between detection signals of hotspots 962 and 961as the fingertip moves, i.e., the y-axis shows the valueSignal difference=detection_signal₉₆₂−detection_signal₉₆₁Thus, at the beginning of the tracked movement the fingertip is fullydetected at both hotspots 961 and 962, and the difference between themis 0. As the fingertip moves toward hotspot 963 it remains fullydetected at hotspot 962 and detection at hotspot 961 is graduallyreduced. The detection values are 8-bit values.

FIG. 78 is a graph that tracks the differences between detection signalsof hotspots 963 and 962 as the fingertip moves, i.e., the y-axis showsthe valueSignal difference=detection_signal₉₆₃−detection_signal₉₆₂Thus, at the beginning of the tracked movement the fingertip is notdetected at hotspot 963 but is fully detected at hotspot 962, and thedifference is −255. As the fingertip moves toward hotspot 963 it remainsfully detected at hotspot 962 and detection at hotspot 963 graduallyincreases.

Although the signal plots in FIGS. 77 and 78 are not linear as afunction of distance from the hotspot, a linear calculation provides aclose estimate as illustrated by the line in each graph marked “Linearestimation”.

FIGS. 79-81 are illustrations showing measurement of errors by aproximity sensor, for a plurality of different objects, in accordancewith an embodiment of the present invention. FIGS. 79-81 show thedifference between object size measured by a proximity sensor and theactual object size, for a plurality of different objects. FIGS. 79-81demonstrate the accuracy of using a linear model for sub-pixelcalculations. FIGS. 79-81 compare (i) the sizes of various objectsmeasured by a proximity sensor of the present invention using linearinterpolation with (ii) the object's actual size. FIG. 79 showscalculated lengths (x-axis) for many objects and the correspondingactual lengths of the objects. For all objects except the tape shown inFIG. 70 the calculated length matched the actual length. The tape shownin FIG. 70 has a glossy surface curved along the x-axis, which disturbsthe reflection signals compared to reflections off a non-glossy object.This is why the calculated length was not accurate. FIG. 80 showscalculated widths (y-axis) for many objects and the corresponding actualwidths of the objects. For all objects except the tape shown in FIG. 70the calculated width matched the actual width. In this case too, thehighly reflective surface, curved along the y-axis caused an incorrectwidth calculation. FIG. 81 shows calculated heights (z-axis) for manyobjects and the corresponding actual heights of the objects. For allobjects the calculated height matched the actual height. Although thetape shown in FIG. 70 is glossy, it is not curved in the z-axis andtherefore the height calculation is not distorted.

Reference is made to FIG. 82, which is a simplified illustration of asystem for scanning a human body for use in retail clothing applicationsand for generating avatars used in social media apps and in games, inaccordance with an embodiment of the present invention. The systemincludes an array of proximity sensor bars 501 arranged along a rotatingplatform 841 on which a user 843 stands. As platform 841 is rotated,proximity sensor bars 501 capture repeated scans of the person, whichare transmitted to server 703 which converts the scan data into a 3Dmodel as explained hereinabove. In some embodiments, certain keymeasurements are extracted from the scan information, e.g., height,shoulder span, waist size, arm length, inner leg length. In retailapplications, the 3D model or the extracted key measurements are used byserver 703 to search a clothing inventory and find clothes that fit thescanned model. In some embodiments of the invention, this scanningsystem is installed in a store dressing room together with a display 842on which the scanned model is displayed as avatar 844 outfitted in theclothes returned by the search. The user then shops for the clothes ondisplay 842. In certain embodiments of the invention, display 842 is aone-way mirror. In gaming and social media applications, the 3D model orthe extracted key measurements are used to create a personalized avatar844 that the user downloads to his or her phone 833 for use in gamingand social media apps.

Reference is made to FIG. 83, which is an illustration showing asituation in which a second object in a proximity sensor detection planeis not detected. FIG. 83 shows a scenario in which two reflectiveobjects 825 and 826 are in the detection plane of proximity sensor bar510, but because the two objects are aligned along the path of theemitter beams illustrated by beam 421, object 826 farther from proximitysensor bar 510 is not detected. FIG. 83 shows emitter beam 421 andreflection 422 off object 825. Emitter beam 421 would have to extend 425beyond object 825 in order to encounter object 826.

Reference is made to FIG. 84, which is an illustration showing how twoproximity sensor bars along different edges of a detection plane areused to solve the problem of FIG. 83, in accordance with an embodimentof the present invention. FIG. 84 shows how a second proximity sensorbar 511 placed along a second edge of the detection plane solves theproblem of FIG. 83. The two objects 825 and 826 reflect differentemitter beams 427 and 426 and both objects are detected. Although FIG.84 shows proximity sensor bars 510 and 511 placed along adjacent edgesof screen 816, in other embodiments the two proximity sensor bars areplaced along opposite edges of screen 816, along three edges or alongall four edges.

Reference is made to FIG. 85, which is a simplified illustration showingsome of the projected and reflected light paths used by the twoproximity sensor bars of FIG. 84 along different edges of the detectionplane, in accordance with an embodiment of the present invention. FIG.85 shows the emitter beams and paths of maximum detected reflections forproximity sensor bar 510 on the right, and for proximity sensor bar 511on the left. Specifically, the lines perpendicular to the screen edgefrom which they exit represent the emitters beam paths, and the linestilted in relation to that screen edge represent paths of maximumdetected reflections. Only the set of forward maximum detectedreflection paths is shown in order not to clutter the figure. Becausethe detection area is rectangular, the paths of maximum detectedreflections are at different angles in relation to each proximitysensor, as each sensor is designed to maximize the number of hotspots inthe screen area and distribute those hotspots as evenly as possiblewithin the screen area.

Reference is made to FIG. 86, which is a simplified illustration showinga touch screen and two proximity sensor bars for placement alongadjacent edges thereof, in accordance with an embodiment of the presentinvention. FIG. 86 shows how proximity sensor bars 510 and 511 are addedto screen 816. In some embodiments, screen 816 is an electronic paperdisplay and in other embodiments it is not a display but rather an inputsurface such as a touch pad with or without icons printed thereon.

Reference is made to FIG. 87, which is an exploded view of a touchscreen assembly featuring two proximity sensor bars 510 and 511 alongadjacent edges of the screen, in accordance with an embodiment of thepresent invention. FIG. 87 shows top cover 830 and bottom cover 831,display screen 816, proximity sensor bars 510 and 511 and light guide309. Light guide 309 directs light beams from proximity sensor bars 510and 511 over and across the exposed upper surface of display screen 816and similarly directs reflections of those beams by objects touching thedisplay screen back onto proximity sensor bars 510 and 511. In someembodiments, one shared processor synchronizes activations of theemitters and detectors of both proximity sensor bars 510 and 511.

Reference is made to FIG. 88, which is a cutaway view of the touchscreen assembly of FIG. 87, showing paths of projected and reflectedlight beams into and out of one of the proximity sensor bars, inaccordance with an embodiment of the present invention. FIG. 88 showsthe paths of light beams 427 from proximity sensor bar 510 through lightguide 309 over and across the exposed upper surface of display screen816.

Reference is made to FIG. 89, which shows three simplified flow chartsof methods of activating two proximity sensor bars along different edgesof a touch surface, in accordance with an embodiment of the presentinvention. FIG. 89 shows flowcharts of alternative activation schemesfor proximity sensor bars 510 and 511. Steps 1010-1012 describe a firstscheme whereby only one of proximity sensor bars 510 and 511 isactivated until an object is detected, at which point both proximitysensor bars are activated. Steps 1013-1015 describe a second schemewhereby only one of proximity sensor bars 510 and 511 is activated untiltwo objects are detected, at which point both proximity sensor bars areactivated. This second scheme uses less power than the first scheme whenonly one object is detected. In most cases in which two objects touchthe screen the objects aren't aligned along an emitter beam path, so thesecond scheme waits until multiple objects are detected beforeactivating the second proximity sensor bar. Steps 1016-1020 describe athird scheme whereby only the shorter of proximity sensor bars 510 and511 is activated until one object is detected, at which point only thelonger proximity sensor bar is activated. The shorter proximity sensorbar activates fewer components to scan the display area thereby reducingpower consumption. However, the longer proximity sensor bar has betterresolution so it is activated to track the object. When the longerproximity sensor bar detects two objects, both proximity sensor bars areactivated.

Reference is made to FIG. 90, which is an illustration of a wirelessinput-output system, in accordance with an embodiment of the presentinvention. FIG. 90 shows smartphone 833 displaying GUI 901. The GUI istransmitted over a first wireless communication channel 630, e.g., usingthe Wi-Fi Direct standard used by Miracast-certified devices, to TV 832.Thus, an enlarged GUI 902 on TV 832 includes the same elements as GUI901.

Proximity sensor bar 501 is positioned along the bottom edge of TV 832.This proximity sensor bar has been described hereinabove, inter alia,with reference to FIGS. 10-13. In some embodiments of the invention,proximity sensor bar 501 features a battery mounted in its housing thatpowers proximity sensor bar 501. The detection plane of this sensor baris along the surface of TV 832 such that user gestures performed on GUI902 are detected by proximity sensor bar 501. FIG. 91 shows wirelesscommunications chip 512 mounted in proximity sensor bar 501. Wirelesscommunications chip 512 sends the detected user gestures, and/or thecoordinates of objects detected touching TV 832, to dongle 834 pluggedinto smartphone 833, over a second wireless communication channel 631,e.g., using the Bluetooth® wireless technology standard. BLUETOOTH is aregistered trademark of Bluetooth SIG, Inc. Corporation. Dongle 834transmits the gesture data to the smartphone's operating system. Thesmartphone's operating system responds to the gestures as if they wereperformed on smartphone 833. In this way, TV 832 serves as a remoteinput and output terminal for smartphone 833. In some embodiments of theinvention mobile phone 833 includes an embedded communications chip forenabling wireless communication channel 631 and dongle 834 is not used.

Wireless channel 630 is enabled by communications chips in TV 832, or byan adapter that plugs into TV 832, e.g., via High-Definition MultimediaInterface (HDMI) or Universal Serial Bus (USB) ports on the TV. Thedescribed system with smartphone 833 and TV 832 is exemplary. In otherembodiments of the invention smartphone 833 is replaced by a server, alaptop or a tablet. Similarly, in certain embodiments of the inventionTV 832 is replaced by a projector or other display device. For example,when TV 832 is replaced by a projector, proximity sensor bar 501 isplaced along an edge of the surface on which GUI 902 is projected todetect user interactions with that GUI.

In certain embodiments of the invention, a second proximity sensor baris placed along a second edge of GUI 902, either adjacent to oropposite, the edge along which proximity sensor bar 501 is placed, asdescribed hereinabove with reference to FIGS. 84-88. In this case, bothproximity sensors use a single, shared wireless communications chip 512to communicate with smartphone 833.

Reference is made to FIG. 91, which is a simplified block diagram of awireless input-output system featuring a mobile phone, a TV and aproximity sensor bar, in accordance with an embodiment of the presentinvention. FIG. 91 shows the components in each of the units in thesystem of FIG. 90. TV 832 features multiple inputs, including Wi-Fi, andcomponents for audio-visual processing and output. Smartphone 833,labeled “Mobile Phone” in FIG. 91, features an applications processorand a baseband processor, in addition to components that enableBluetooth and Wi-Fi input and output. Proximity sensor 501 includesemitters 101-110, detectors 201-211, controller 701 for activating theemitters and detectors and extracting object location information fromthe detector outputs, and communications processor 512 for sending theextracted information to smartphone 833. Video data is streamed fromsmartphone 833 to TV 832 over Wi-Fi channel 630. Gesture input isdetected by proximity sensor 501 and is transmitted to smartphone 833over Bluetooth channel 631. The applications processor in smartphone 833controls the video output in response to the detected gestures.

Reference is made to FIG. 92, which is a simplified block diagram of awireless input-output system featuring a server, a thin client and aproximity sensor bar, in accordance with an embodiment of the presentinvention. FIG. 92 shows another wireless input-output system in whichsmartphone 833 is replaced by a home server, and TV 832 is replaced by athin client. Video and image data, including user interface screens, isstreamed from the home server to the thin client over wireless LAN.Gesture input is detected by proximity sensor 501 and is transmitted tothe home server over wireless channel 631. The home server controls thevideo and image output in response to the detected gestures.

Reference is made to FIG. 93, which is a simplified block diagram of awireless input-output system featuring a thin client and a proximitysensor bar, in accordance with an embodiment of the present invention.FIG. 93 shows a wireless input-output system similar to that of FIG. 92,except that the server of FIG. 92 is incorporated into the housing ofproximity sensor 501. This provides a powerful, versatile unit that runsapplications and generates data in response to gestures it detects. Thisunit streams graphic data to one or more thin clients over wireless LAN.The unit is compact, making it easy to travel with, and it is easy toconnect to any display that supports wireless input. The user places theunit adjacent to the display to detect gestures on the display.Alternatively, the user can place the unit anywhere, e.g., in the user'slap, and detect gestures performed in the unit's detection plane tocontrol the output on the display. The proximity sensors in the systemsof FIGS. 90-92 can likewise be placed anywhere, e.g., in the user's lap,and detect gestures performed in the proximity sensor's detection planeto control the output on the TV or thin client as well.

Reference is made to FIGS. 94 and 95, which are simplified illustrationsof a curved display and proximity sensing light-beams used to detectgestures thereon, in accordance with an embodiment of the presentinvention. FIGS. 94 and 95 show a curved reflective surface for use inconjunction with light-based proximity sensor 501 placed along one edge.This feature enables detecting touches on flexible displays even whenthe displays are not flat. FIGS. 94 and 95 indicate locations of emitter104 and detectors 202 and 206 in a proximity sensor 501 (not shown)placed along the bottom edge of curved display screen 836. FIG. 94illustrates the path of light beam 428 emitted by emitter 104. Lightbeam 428 advances across the surface of display screen 836 in segmentsthat follow the contour of the display screen upper surface such thatsome of the light paths are incident upon and reflect off of the curvedupper surface while crossing the display screen upper surface.

FIG. 95 shows the path of light beam 428 when it encounters a reflectiveobject such as a finger at location 835. Light beam 428 advances acrossthe surface of display screen 836 to location 835 in segments thatfollow the contour of the display screen upper surface such that some ofthe light paths are incident upon and reflect off of the curved uppersurface while crossing the display screen upper surface. Portions of thelight scattered by the object at 835, go back across the surface ofdisplay screen 836 to detectors 202 and 206 in segments that follow thecontour of the display screen upper surface such that some of the lightpaths are incident upon and reflect off of the curved upper surfacewhile heading back over the display screen upper surface.

Reference is made to FIGS. 96-98, which are simplified illustrations ofa flexible display and proximity sensing light-beams used to detectgestures thereon, in accordance with an embodiment of the presentinvention. FIGS. 96-98 show a flexible display for use in conjunctionwith light-based proximity sensor 501 placed along one edge. FIGS. 96-98indicate locations of proximity sensor 501 placed along the bottom edgeof flexible display screen 837. FIG. 96 shows the path of light beam 428emitted by an emitter in proximity sensor 501. Light beam 428 advancesacross the curved top portion of the display in segments that follow thecontour of the display screen upper surface such that some of the lightpaths are incident upon and reflect off of the curved upper surfacewhile crossing the display screen upper surface.

FIG. 97 shows the path of light beam 428 when it encounters a reflectiveobject such as finger 819 in the bottom, flat portion of flexibledisplay 837. In this case, the light passes over and across the surfaceof display 837 without being incident upon and reflected off of thedisplay upper surface.

FIG. 98 shows the path of light beam 428 when it encounters finger 819in the upper, curved portion of flexible display 837. Light beam 428advances across the upper, curved portion of flexible display 837 insegments that follow the contour of the display screen upper surfacesuch that some of the light paths are incident upon and reflect off ofthe curved upper surface while crossing the display screen uppersurface. Portions of the light scattered by finger 819, go back acrossthe surface of flexible display 837 to proximity sensor 501 in segmentsthat follow the contour of the display screen upper surface such thatsome of the light paths are incident upon and reflect off of the curvedupper surface while heading back over the display screen upper surface.

Reference is made to FIGS. 99 and 100, which are simplifiedillustrations of a housing holding a retractable display and a proximitysensor that detects gestures on the retractable display when the displayis drawn out of the housing, in accordance with an embodiment of thepresent invention. FIGS. 99 and 100 show a retractable display for usein conjunction with light-based proximity sensor 501, in accordance withembodiments of the present invention. FIG. 99 shows housing 839 in whichretractable display is stored, slot 840 through which the retractabledisplay is raised and lowered, and proximity sensor 501 mounted insidehousing 839. FIG. 100 shows housing 839 with retractable display 838being raised through slot 840. Proximity sensor 501 is configured toproject light beams parallel to the surface of display 838.

Reference is made to FIG. 101, which is a simplified illustration of afirst proximity sensing system that detects objects in multiple paralleldetection planes, in accordance with an embodiment of the presentinvention. FIG. 101 shows an optical assembly for detecting locations ofobjects in multiple parallel spatial planes 970 and 971. The opticalassembly includes reflectance-based sensor 501 that emits light intodetection plane 970 and detects reflections of the emitted light,reflected by an object located in detection plane 970. A lightre-director 827, positioned away from sensor 501, re-directs the lightemitted by sensor 501 into spatial plane 971, parallel to detectionplane 970. Light re-director 827 is illustrated as two mirrors. When theobject is located in spatial plane 971, light re-director 827 re-directslight reflected by the object into detection plane 970 of sensor 501.

Reference is made to FIGS. 102 and 103, which are simplifiedillustrations of mapping detections in different detection planes in theproximity sensing system of FIG. 101, in accordance with an embodimentof the present invention. FIG. 102 shows how locations of objects in thedifferent parallel spatial planes 970 and 971 are detected by aprocessor connected to sensor 501. Spatial plane 971 is mapped as anextension of plane 970. Object coordinates located within thisextension, based on the y-coordinate are known to be located in parallelplane 971. Thus, the processor detects one or more virtual locations ofthe object within detection plane 970 of sensor 501, based on lightreflected by the object that is re-directed to detection plane 970 ofsensor 501 and detected by sensor 501. The processor transforms the oneor more virtual locations of the object within detection plane 970 ofsensor 501 to corresponding one or more real locations of the objectwithin one or more spatial planes 971 parallel to detection plane 970 ofsensor 501, based on the position of light re-director 827 relative tosensor 501.

FIG. 103 shows two hotspot locations 930 and 931 for emitter-detectorpairs 101-201 and 101-202 of sensor 501. Hotspot location 930 is inplane 970 and hotspot location 931 is in plane 971, but, as explainedabove, both locations are mapped to detection plane 970. Light beam 429is reflected as beams 434 and 435 by an object at hotspots 930 and 931and these reflections are maximally detected by detectors 201 and 202,respectively.

In some embodiments of the invention, sensor 501 includes interleavedemitters and detectors as illustrated in FIG. 10. In other embodimentsof the invention, sensor 501 is a time-of-flight proximity sensor whosetime-of-flight beams are redirected by light re-director 827. In yetother embodiments of the invention, sensor 501 includes two cameras thatcapture images of objects in parallel plane 971 and identify theobject's location by the triangulating the object in the camera images.

Reference is made to FIG. 104, which is a simplified illustration of asecond proximity sensing system that detects objects in multipleparallel detection planes, in accordance with an embodiment of thepresent invention. FIG. 104 shows an embodiment of an optical assemblyfor detecting locations of objects in multiple parallel spatial planes970 and 971 in which light re-directors 827 and 828 split light beam 429from sensor 501. In this embodiment, a first portion of emitted lightbeam 429 is directed by light re-director 827 to plane 970 above sensor501, and a second portion of emitted light beam 429 is directed by lightre-director 828 beneath sensor 501. In this embodiment, an objectreflecting the first portion of light beam 429, above sensor 501, and anobject reflecting the second portion of light beam 429, beneath sensor501, are the same distance from sensor 501 and will generate similardetections. In order to distinguish an object above sensor 501 from anobject beneath sensor 501, a third light re-director 829 is provided tofurther redirect light beneath sensor 501 into plane 971. Thus, objectsin plane 971 are further away from sensor 501 and generate differentreflection patterns than objects in plane 970.

Reference is made to FIGS. 105 and 106, which are simplifiedillustrations of a keyboard using the proximity sensing system of FIG.104 to detect objects above the keys and also detect depressed keys, inaccordance with an embodiment of the present invention. FIGS. 105 and106 show a keyboard application that includes keys 850 and 851. Sensor501 detects movement of objects above keys 850 and 851. This movement isinterpreted as trackpad input. Each key features a rod 852 beneath thekey connected to a block 854 or 855. Blocks 854 and 855 in their restingposition are suspended above plane 971 by springs 853. Sensor 501detects when key 850 or 851 is depressed as this movement lowersrespective block 854 or 855 into detection plane 971. The resolution ofobject locations above the keys must be finer than that beneath thekeys: there are only few discrete locations into which blocks 854 or 855can be lowered into plane 971, whereas a finger above the keys can movefreely within plane 970. In FIG. 105 no keys are depressed, and in FIG.106 key 850 is depressed.

Reference is made to FIGS. 107 and 108, which are simplifiedillustrations of mapping detections above the keys, and detections ofdepressed keys, in the keyboard of FIGS. 105 and 106, in accordance withan embodiment of the present invention. FIG. 107 shows how detections inplane 970, above the keys in the keyboard of FIGS. 105 and 106, aredetected as nearer sensor 501 than detections of depressed keys in plane971.

FIG. 108 shows detections in planes 970 and 971 in the keyboard of FIGS.105 and 106. As illustrated in FIGS. 105 and 106, rods 852 block some ofthe light underneath the keys, and reflections off these rods will bedetected as objects in plane 970. In FIG. 108, these detections areindicated as objects 856-861. Furthermore, as illustrated in FIGS. 105and 106, rods 852 block some light from reaching plane 971 and blocksome reflections in plane 971 from reaching sensor 501. These problemsare minimized in several ways. First, rods 852 are thin and coloredblack to minimize the amount of reflection they generate. Second, rods852 reflect light even in their resting position, creating a baselineamount of detected reflections. Therefore, objects above the keys, inplane 970, are detected by an increase in detected reflections above thebaseline. Third, detections of lowered keys in plane 971 do not requirehigh resolution detection to identify which key is being depressed.Thus, even with rods 852 causing some interference, keys lowered intoplane 971 are detected and identified.

In the foregoing specification, the invention has been described withreference to specific exemplary embodiments thereof. It will, however,be evident that various modifications and changes may be made to thespecific exemplary embodiments without departing from the broader spiritand scope of the invention. Accordingly, the specification and drawingsare to be regarded in an illustrative rather than a restrictive sense.

The invention claimed is:
 1. An interactive mid-air display comprising:a display that presents a graphical user interface (GUI); opticsconfigured in accordance with said display projecting and rotating theGUI presented on said display above said display such that the GUI isvisible in-air in a plane rotated away from said display; a reflectancesensor comprising: a plurality of light emitters projecting a pluralityof evenly spaced beams towards the projected and rotated GUI; aplurality of light detectors detecting reflections of the evenly spacedbeams by objects interacting with the projected GUI; and a lensstructure positioned in relation to said light detectors maximizingdetection of light at each detector for light entering the lensstructure at a respective location along the lens structure at aspecific angle of incidence θ, whereby for each emitter-detector paircomprising one of said light emitters and one of said light detectors,maximum detection of light projected by the emitter of the pair,reflected by an object and detected by the detector of the pair,corresponds to the object being at a specific 2D location in theprojected and rotated GUI, in accordance with the locations of the oneemitter and one detector and the angle θ; and a processor connected tothe display, receiving outputs from said sensor, mapping detections oflight for a plurality of emitter-detector pairs to their corresponding2D locations in the projected and rotated GUI, and translating themapped locations to coordinates on said display.
 2. A flex sensorcomprising: a housing; a plurality of light emitters mounted in saidhousing, which, when said housing is placed in a user's open hand,projects light beams in a detection plane above and across the extendedmedial four digits of the hand; a plurality of light detectors mountedin said housing, detecting reflections of the projected beams when thedigits are flexed; and a lens structure positioned in relation to saidlight emitters and to said light detectors, directing emitter light outof the lens structure at a specific angle φ, and maximizing detection oflight at each detector for light entering the lens structure at arespective location along the lens structure at a specific angle ofincidence θ, substantially different than angle φ, whereby for eachemitter-detector pair comprising one of said light emitters and one ofsaid light detectors, maximum detection of light projected by theemitter of the pair, reflected by an object and detected by the detectorof the pair, corresponds to the object being at a specific 2D locationin the detection plane, in accordance with the locations of the oneemitter and one detector and the angles θ and φ; and a processorreceiving outputs from said sensor, mapping detections of light for aplurality of emitter-detector pairs to corresponding flexed digits, andtranslating the flexed digits to input for a system controlled by theuser.
 3. The flex sensor of claim 2, wherein different arcs of digitflexion correspond to different emitter-detector pairs.
 4. The flexsensor of claim 3, wherein greater arcs of digit flexion correspond toemitter-detector pairs characterized by a shorter distance between theemitter and the detector of the pair.
 5. The flex sensor of claim 2,wherein said processor translates different flexed digits to differentinputs.
 6. The flex sensor of claim 2, wherein said processor translatesdifferent combinations of flexed digits to different inputs.
 7. The flexsensor of claim 2, wherein said processor translates the mapped flexeddigits to commands for a virtual reality (VR) system.
 8. The flex sensorof claim 7, wherein said processor translates a detected grab gesturecomprising flexing extended digits, to a command to pick up a virtualobject in the VR system.
 9. The flex sensor of claim 8, wherein saidprocessor translates a detected release gesture comprising extendingflexed digits, to a command to release a virtual object in the VRsystem.
 10. A reflectance sensor for a virtual display systemcomprising: a housing; a plurality of light emitters mounted in saidhousing, projecting a plurality of light beams across a detection plane;a plurality of light detectors mounted in said housing, detectingreflections of the projected light beams by an object in the detectionplane; and a lens structure positioned in relation to said lightdetectors maximizing detection of light at each detector for lightentering the lens structure at a respective location along the lensstructure at a specific angle of incidence θ, whereby for eachemitter-detector pair comprising one of said light emitters and one ofsaid light detectors, maximum detection of light projected by theemitter of the pair, reflected by an object and detected by the detectorof the pair, corresponds to the object being at a specific 2D locationin the detection plane, in accordance with the locations of the oneemitter and one detector and the angle θ; a processor receiving outputsfrom said detectors and identifying locations of objects in thedetection plane based on detections of light for a plurality ofemitter-detector pairs; and a wireless transmitter for sending theidentified object locations to a head-up display that presents a GUIaligned with the detection plane, enabling the head-up display to mapthe identified object locations as interactions with correspondingelements in the GUI.
 11. The reflectance sensor of claim 10, whereinsaid housing comprises an exposed graphic to facilitate aligning the GUIwith the sensor detection plane in the head-up display viewer's field ofview.
 12. The reflectance sensor of claim 10, wherein said reflectancesensor is mounted in a wrist-worn accessory, and wherein the detectionplane is along the forearm or wrist adjacent to said sensor.